Predict Bike Sharing Demand with AutoGluon Template

Project: Predict Bike Sharing Demand with AutoGluon

This notebook is a template with each step that you need to complete for the project.

Please fill in your code where there are explicit ? markers in the notebook. You are welcome to add more cells and code as you see fit.

Once you have completed all the code implementations, please export your notebook as a HTML file so the reviews can view your code. Make sure you have all outputs correctly outputted.

File-> Export Notebook As... -> Export Notebook as HTML

There is a writeup to complete as well after all code implememtation is done. Please answer all questions and attach the necessary tables and charts. You can complete the writeup in either markdown or PDF.

Completing the code template and writeup template will cover all of the rubric points for this project.

The rubric contains "Stand Out Suggestions" for enhancing the project beyond the minimum requirements. The stand out suggestions are optional. If you decide to pursue the "stand out suggestions", you can include the code in this notebook and also discuss the results in the writeup file.

Step 1: Create an account with Kaggle

Create Kaggle Account and download API key

Below is example of steps to get the API username and key. Each student will have their own username and key.

  1. Open account settings. kaggle1.png kaggle2.png
  2. Scroll down to API and click Create New API Token. kaggle3.png kaggle4.png
  3. Open up kaggle.json and use the username and key. kaggle5.png

Step 2: Download the Kaggle dataset using the kaggle python library

Open up Sagemaker Studio and use starter template

  1. Notebook should be using a ml.t3.medium instance (2 vCPU + 4 GiB)
  2. Notebook should be using kernal: Python 3 (MXNet 1.8 Python 3.7 CPU Optimized)

Install packages

In [1]:
!pip install -U pip
!pip install -U setuptools wheel
!pip install -U "mxnet<2.0.0" bokeh==2.0.1
!pip install autogluon --no-cache-dir
# Without --no-cache-dir, smaller aws instances may have trouble installing
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (22.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (59.5.0)
Collecting setuptools
  Using cached setuptools-62.3.2-py3-none-any.whl (1.2 MB)
Requirement already satisfied: wheel in /usr/local/lib/python3.7/site-packages (0.37.1)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 59.5.0
    Uninstalling setuptools-59.5.0:
      Successfully uninstalled setuptools-59.5.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
autogluon-text 0.4.1 requires setuptools<=59.5.0, but you have setuptools 62.3.2 which is incompatible.
Successfully installed setuptools-62.3.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Requirement already satisfied: mxnet<2.0.0 in /usr/local/lib/python3.7/site-packages (1.9.1)
Requirement already satisfied: bokeh==2.0.1 in /usr/local/lib/python3.7/site-packages (2.0.1)
Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (1.21.6)
Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (21.3)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (2.8.2)
Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (5.4.1)
Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (6.1)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (4.0.1)
Requirement already satisfied: Jinja2>=2.7 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (3.0.3)
Requirement already satisfied: pillow>=4.0 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (9.0.1)
Requirement already satisfied: graphviz<0.9.0,>=0.8.1 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (0.8.4)
Requirement already satisfied: requests<3,>=2.20.0 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (2.22.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from Jinja2>=2.7->bokeh==2.0.1) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/site-packages (from packaging>=16.8->bokeh==2.0.1) (3.0.6)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil>=2.1->bokeh==2.0.1) (1.16.0)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2021.10.8)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Requirement already satisfied: autogluon in /usr/local/lib/python3.7/site-packages (0.4.1)
Requirement already satisfied: autogluon.text==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1)
Requirement already satisfied: autogluon.vision==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1)
Requirement already satisfied: autogluon.core[all]==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1)
Requirement already satisfied: autogluon.features==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1)
Requirement already satisfied: autogluon.tabular[all]==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1)
Requirement already satisfied: autogluon.common==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (0.4.1)
Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2.22.0)
Requirement already satisfied: scipy<1.8.0,>=1.5.4 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.7.3)
Requirement already satisfied: numpy<1.23,>=1.21 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.21.6)
Requirement already satisfied: pandas<1.4,>=1.2.5 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.3.4)
Requirement already satisfied: dask<=2021.11.2,>=2021.09.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2021.11.2)
Requirement already satisfied: tqdm>=4.38.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (4.64.0)
Requirement already satisfied: scikit-learn<1.1,>=1.0.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.0.1)
Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.20.17)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (3.5.0)
Requirement already satisfied: distributed<=2021.11.2,>=2021.09.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2021.11.2)
Requirement already satisfied: ray<1.11,>=1.10 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.10.0)
Requirement already satisfied: psutil<5.9,>=5.7.3 in /usr/local/lib/python3.7/site-packages (from autogluon.features==0.4.1->autogluon) (5.8.0)
Requirement already satisfied: networkx<3.0,>=2.3 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (2.6.3)
Requirement already satisfied: torch<1.11,>=1.0 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.10.2)
Requirement already satisfied: catboost<1.1,>=1.0 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.0.6)
Requirement already satisfied: fastai<2.6,>=2.3.1 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (2.5.6)
Requirement already satisfied: xgboost<1.5,>=1.4 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.4.2)
Requirement already satisfied: lightgbm<3.4,>=3.3 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (3.3.2)
Requirement already satisfied: autogluon-contrib-nlp==0.0.1b20220208 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.0.1b20220208)
Requirement already satisfied: scikit-image<0.20.0,>=0.19.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.19.2)
Requirement already satisfied: nptyping<1.5.0,>=1.4.4 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (1.4.4)
Requirement already satisfied: smart-open<5.3.0,>=5.2.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (5.2.1)
Requirement already satisfied: torchmetrics<0.8.0,>=0.7.2 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.7.3)
Requirement already satisfied: pytorch-lightning<1.7.0,>=1.5.10 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (1.6.3)
Collecting setuptools<=59.5.0
  Downloading setuptools-59.5.0-py3-none-any.whl (952 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 952.4/952.4 kB 162.3 MB/s eta 0:00:00
Requirement already satisfied: omegaconf<2.2.0,>=2.1.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (2.1.2)
Requirement already satisfied: timm<0.6.0 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.5.4)
Requirement already satisfied: transformers<4.17.0,>=4.16.2 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (4.16.2)
Requirement already satisfied: fairscale<0.5.0,>=0.4.5 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.4.6)
Requirement already satisfied: Pillow<9.1.0,>=9.0.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (9.0.1)
Requirement already satisfied: sentencepiece<0.2.0,>=0.1.95 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.1.95)
Requirement already satisfied: gluoncv<0.10.6,>=0.10.5 in /usr/local/lib/python3.7/site-packages (from autogluon.vision==0.4.1->autogluon) (0.10.5.post0)
Requirement already satisfied: yacs>=0.1.6 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.1.8)
Requirement already satisfied: protobuf in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (3.19.1)
Requirement already satisfied: regex in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2022.4.24)
Requirement already satisfied: flake8 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (4.0.1)
Requirement already satisfied: sacremoses>=0.0.38 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.0.53)
Requirement already satisfied: pyarrow in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (6.0.1)
Requirement already satisfied: sacrebleu in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.1.0)
Requirement already satisfied: tokenizers>=0.9.4 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.12.1)
Requirement already satisfied: contextvars in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.4)
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (1.16.0)
Requirement already satisfied: plotly in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (5.4.0)
Requirement already satisfied: graphviz in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (0.8.4)
Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (5.4.1)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2021.11.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (0.11.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (21.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (3.0.3)
Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.4.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.2.0)
Requirement already satisfied: fastcore<1.5,>=1.3.27 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.4.3)
Requirement already satisfied: torchvision>=0.8.2 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.11.3)
Requirement already satisfied: spacy<4 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (2.3.7)
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (22.1)
Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.2)
Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.0.6)
Requirement already satisfied: autocfg in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (0.0.8)
Requirement already satisfied: portalocker in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (2.3.2)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (4.5.4.60)
Requirement already satisfied: wheel in /usr/local/lib/python3.7/site-packages (from lightgbm<3.4,>=3.3->autogluon.tabular[all]==0.4.1->autogluon) (0.37.1)
Requirement already satisfied: typish>=1.7.0 in /usr/local/lib/python3.7/site-packages (from nptyping<1.5.0,>=1.4.4->autogluon.text==0.4.1->autogluon) (1.9.3)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /usr/local/lib/python3.7/site-packages (from omegaconf<2.2.0,>=2.1.1->autogluon.text==0.4.1->autogluon) (4.8)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/site-packages (from pandas<1.4,>=1.2.5->autogluon.core[all]==0.4.1->autogluon) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/site-packages (from pandas<1.4,>=1.2.5->autogluon.core[all]==0.4.1->autogluon) (2021.3)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (4.0.1)
Requirement already satisfied: pyDeprecate<0.4.0,>=0.3.1 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.3.2)
Requirement already satisfied: tensorboard>=2.2.0 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.9.0)
Requirement already satisfied: redis>=3.5.0 in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.3.1)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.5.1)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (3.7.0)
Requirement already satisfied: grpcio>=1.28.1 in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.46.3)
Requirement already satisfied: attrs in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (21.2.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (2021.11.2)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (1.3.0)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (2.13.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn<1.1,>=1.0.0->autogluon.core[all]==0.4.1->autogluon) (3.0.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/site-packages (from scikit-learn<1.1,>=1.0.0->autogluon.core[all]==0.4.1->autogluon) (1.1.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/site-packages (from transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (4.2.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/site-packages (from transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (0.6.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (0.5.0)
Requirement already satisfied: botocore<1.24.0,>=1.23.17 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (1.23.17)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (3.0.6)
Requirement already satisfied: setuptools-scm>=4 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (6.3.2)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (4.28.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (0.11.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (2021.10.8)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (1.25.11)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/site-packages (from fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (3.8.1)
Requirement already satisfied: locket in /usr/local/lib/python3.7/site-packages (from partd>=0.3.10->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.0)
Requirement already satisfied: async-timeout>=4.0.2 in /usr/local/lib/python3.7/site-packages (from redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.0.2)
Requirement already satisfied: deprecated>=1.2.3 in /usr/local/lib/python3.7/site-packages (from redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.2.13)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/site-packages (from importlib-metadata->transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (3.6.0)
Requirement already satisfied: tomli>=1.0.0 in /usr/local/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib->autogluon.core[all]==0.4.1->autogluon) (1.2.2)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (3.0.6)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.1.3)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.7)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.7.7)
Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.5)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (2.0.6)
Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.9.1)
Requirement already satisfied: thinc<7.5.0,>=7.4.1 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (7.4.5)
Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.4.6)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.0.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.6.6)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.0.2)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (3.3.4)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.8.1)
Requirement already satisfied: heapdict in /usr/local/lib/python3.7/site-packages (from zict>=0.1.3->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.1)
Requirement already satisfied: immutables>=0.9 in /usr/local/lib/python3.7/site-packages (from contextvars->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.17)
Requirement already satisfied: pyflakes<2.5.0,>=2.4.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.4.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.6.1)
Requirement already satisfied: pycodestyle<2.9.0,>=2.8.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.8.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from jinja2->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/site-packages (from jsonschema->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/site-packages (from jsonschema->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (5.7.1)
Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.7/site-packages (from plotly->catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (8.0.1)
Requirement already satisfied: colorama in /usr/local/lib/python3.7/site-packages (from sacrebleu->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.4.3)
Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.7/site-packages (from sacrebleu->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.8.9)
Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.7/site-packages (from deprecated>=1.2.3->redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.14.1)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (4.7.2)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (5.1.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.3.1)
Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (0.13.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (6.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.7.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.3.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.12)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.2.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (3.2.0)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 62.3.2
    Uninstalling setuptools-62.3.2:
      Successfully uninstalled setuptools-62.3.2
Successfully installed setuptools-59.5.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Setup Kaggle API Key

In [2]:
# create the .kaggle directory and an empty kaggle.json file
!mkdir -p /root/.kaggle
!touch /root/.kaggle/kaggle.json
!chmod 600 /root/.kaggle/kaggle.json
In [26]:
# Fill in your user name and key from creating the kaggle account and API token file
!pip install python-dotenv
import json
from dotenv import dotenv_values

CONFIG = dotenv_values('.env')

kaggle_username = CONFIG['KAGGLE_USERNAME']
kaggle_key = CONFIG['KAGGLE_KEY']

# Save API token the kaggle.json file
with open("/root/.kaggle/kaggle.json", "w") as f:
    f.write(json.dumps({"username": kaggle_username, "key": kaggle_key}))
Requirement already satisfied: python-dotenv in /usr/local/lib/python3.7/site-packages (0.20.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Download and explore dataset

Go to the bike sharing demand competition and agree to the terms

kaggle6.png

In [4]:
# Download the dataset, it will be in a .zip file so you'll need to unzip it as well.
!pip install kaggle
!kaggle competitions download -c bike-sharing-demand
# If you already downloaded it you can use the -o command to overwrite the file
!unzip -o bike-sharing-demand.zip
Collecting kaggle
  Using cached kaggle-1.5.12-py3-none-any.whl
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.25.11)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.16.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/site-packages (from kaggle) (4.64.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/site-packages (from kaggle) (2021.10.8)
Collecting python-slugify
  Using cached python_slugify-6.1.2-py2.py3-none-any.whl (9.4 kB)
Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from kaggle) (2.22.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/site-packages (from kaggle) (2.8.2)
Collecting text-unidecode>=1.3
  Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (3.0.4)
Installing collected packages: text-unidecode, python-slugify, kaggle
Successfully installed kaggle-1.5.12 python-slugify-6.1.2 text-unidecode-1.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
401 - Unauthorized
Archive:  bike-sharing-demand.zip
  inflating: sampleSubmission.csv    
  inflating: test.csv                
  inflating: train.csv               
In [6]:
# import libriries
import pandas as pd
import tqdm as notebook_tqdm
from autogluon.tabular import TabularPredictor
In [7]:
# Create the train dataset in pandas by reading the csv
# Set the parsing of the datetime column so you can use some of the `dt` features in pandas later
train = pd.read_csv('./train.csv', parse_dates=['datetime'])
train.head()
Out[7]:
datetime season holiday workingday weather temp atemp humidity windspeed casual registered count
0 2011-01-01 00:00:00 1 0 0 1 9.84 14.395 81 0.0 3 13 16
1 2011-01-01 01:00:00 1 0 0 1 9.02 13.635 80 0.0 8 32 40
2 2011-01-01 02:00:00 1 0 0 1 9.02 13.635 80 0.0 5 27 32
3 2011-01-01 03:00:00 1 0 0 1 9.84 14.395 75 0.0 3 10 13
4 2011-01-01 04:00:00 1 0 0 1 9.84 14.395 75 0.0 0 1 1
In [8]:
# Simple output of the train dataset to view some of the min/max/varition of the dataset features.
train.describe()
Out[8]:
season holiday workingday weather temp atemp humidity windspeed casual registered count
count 10886.000000 10886.000000 10886.000000 10886.000000 10886.00000 10886.000000 10886.000000 10886.000000 10886.000000 10886.000000 10886.000000
mean 2.506614 0.028569 0.680875 1.418427 20.23086 23.655084 61.886460 12.799395 36.021955 155.552177 191.574132
std 1.116174 0.166599 0.466159 0.633839 7.79159 8.474601 19.245033 8.164537 49.960477 151.039033 181.144454
min 1.000000 0.000000 0.000000 1.000000 0.82000 0.760000 0.000000 0.000000 0.000000 0.000000 1.000000
25% 2.000000 0.000000 0.000000 1.000000 13.94000 16.665000 47.000000 7.001500 4.000000 36.000000 42.000000
50% 3.000000 0.000000 1.000000 1.000000 20.50000 24.240000 62.000000 12.998000 17.000000 118.000000 145.000000
75% 4.000000 0.000000 1.000000 2.000000 26.24000 31.060000 77.000000 16.997900 49.000000 222.000000 284.000000
max 4.000000 1.000000 1.000000 4.000000 41.00000 45.455000 100.000000 56.996900 367.000000 886.000000 977.000000
In [9]:
# Create the test pandas dataframe in pandas by reading the csv, remember to parse the datetime!
test = pd.read_csv('./test.csv', parse_dates=['datetime'])
test.head()
Out[9]:
datetime season holiday workingday weather temp atemp humidity windspeed
0 2011-01-20 00:00:00 1 0 1 1 10.66 11.365 56 26.0027
1 2011-01-20 01:00:00 1 0 1 1 10.66 13.635 56 0.0000
2 2011-01-20 02:00:00 1 0 1 1 10.66 13.635 56 0.0000
3 2011-01-20 03:00:00 1 0 1 1 10.66 12.880 56 11.0014
4 2011-01-20 04:00:00 1 0 1 1 10.66 12.880 56 11.0014
In [10]:
# Same thing as train and test dataset
submission = pd.read_csv('./sampleSubmission.csv')
submission.head()
Out[10]:
datetime count
0 2011-01-20 00:00:00 0
1 2011-01-20 01:00:00 0
2 2011-01-20 02:00:00 0
3 2011-01-20 03:00:00 0
4 2011-01-20 04:00:00 0
In [11]:
# check the columns used for training
train.columns
Out[11]:
Index(['datetime', 'season', 'holiday', 'workingday', 'weather', 'temp',
       'atemp', 'humidity', 'windspeed', 'casual', 'registered', 'count'],
      dtype='object')
In [12]:
# check the columns used for testing
test.columns
Out[12]:
Index(['datetime', 'season', 'holiday', 'workingday', 'weather', 'temp',
       'atemp', 'humidity', 'windspeed'],
      dtype='object')
In [13]:
# check the size of each dataset
train.shape, test.shape
Out[13]:
((10886, 12), (6493, 9))

Step 3: Train a model using AutoGluon’s Tabular Prediction

Requirements:

  • We are predicting count, so it is the label we are setting.
  • Ignore casual and registered columns as they are also not present in the test dataset.
  • Use the root_mean_squared_error as the metric to use for evaluation.
  • Set a time limit of 10 minutes (600 seconds).
  • Use the preset best_quality to focus on creating the best model.
In [14]:
predictor = TabularPredictor(
    label='count',
    problem_type='regression',
    eval_metric='root_mean_squared_error',
    learner_kwargs={"ignored_columns": ["casual", "registered"]},
).fit(
    train_data=train,
    time_limit=600,
    presets='best_quality'
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_115203/"
Presets specified: ['best_quality']
Beginning AutoGluon training ... Time limit = 600s
AutoGluon will save models to "AutogluonModels/ag-20220521_115203/"
AutoGluon Version:  0.4.1
Python Version:     3.7.10
Operating System:   Linux
Train Data Rows:    10886
Train Data Columns: 11
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
	Available Memory:                    2705.09 MB
	Train Data (Original)  Memory Usage: 0.78 MB (0.0% of available memory)
	Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
	Stage 1 Generators:
		Fitting AsTypeFeatureGenerator...
			Note: Converting 2 features to boolean dtype as they only contain 2 unique values.
	Stage 2 Generators:
		Fitting FillNaFeatureGenerator...
	Stage 3 Generators:
		Fitting IdentityFeatureGenerator...
		Fitting DatetimeFeatureGenerator...
	Stage 4 Generators:
		Fitting DropUniqueFeatureGenerator...
	Types of features in original data (raw dtype, special dtypes):
		('datetime', []) : 1 | ['datetime']
		('float', [])    : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])      : 5 | ['season', 'holiday', 'workingday', 'weather', 'humidity']
	Types of features in processed data (raw dtype, special dtypes):
		('float', [])                : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])                  : 3 | ['season', 'weather', 'humidity']
		('int', ['bool'])            : 2 | ['holiday', 'workingday']
		('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
	0.4s = Fit runtime
	9 features in original data used to generate 13 features in processed data.
	Train Data (Processed) Memory Usage: 0.98 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.52s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
	To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 399.55s of the 599.48s of remaining time.
	-101.5462	 = Validation score   (root_mean_squared_error)
	0.03s	 = Training   runtime
	0.1s	 = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 399.17s of the 599.1s of remaining time.
	-84.1251	 = Validation score   (root_mean_squared_error)
	0.03s	 = Training   runtime
	0.1s	 = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 398.8s of the 598.72s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 11:52:10,264	WARNING services.py:1866 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=0.87gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.
	-131.4609	 = Validation score   (root_mean_squared_error)
	63.86s	 = Training   runtime
	6.67s	 = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 323.38s of the 523.31s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-131.0542	 = Validation score   (root_mean_squared_error)
	27.5s	 = Training   runtime
	1.28s	 = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 292.37s of the 492.29s of remaining time.
	-116.5443	 = Validation score   (root_mean_squared_error)
	10.81s	 = Training   runtime
	0.85s	 = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 277.91s of the 477.84s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-130.5106	 = Validation score   (root_mean_squared_error)
	201.26s	 = Training   runtime
	0.17s	 = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 73.22s of the 273.15s of remaining time.
	-124.5881	 = Validation score   (root_mean_squared_error)
	4.85s	 = Training   runtime
	0.52s	 = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 65.21s of the 265.14s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-138.0336	 = Validation score   (root_mean_squared_error)
	66.16s	 = Training   runtime
	0.45s	 = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 196.13s of remaining time.
	-84.1251	 = Validation score   (root_mean_squared_error)
	0.64s	 = Training   runtime
	0.0s	 = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 195.4s of the 195.38s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-60.1464	 = Validation score   (root_mean_squared_error)
	53.09s	 = Training   runtime
	3.42s	 = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 138.3s of the 138.28s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-55.0493	 = Validation score   (root_mean_squared_error)
	26.54s	 = Training   runtime
	0.24s	 = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 108.93s of the 108.91s of remaining time.
	-53.4369	 = Validation score   (root_mean_squared_error)
	26.17s	 = Training   runtime
	0.59s	 = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 79.64s of the 79.62s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-55.7246	 = Validation score   (root_mean_squared_error)
	70.12s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L2 ... Training model for up to 6.92s of the 6.9s of remaining time.
	-53.841	 = Validation score   (root_mean_squared_error)
	8.21s	 = Training   runtime
	0.59s	 = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -4.61s of remaining time.
	-52.8219	 = Validation score   (root_mean_squared_error)
	0.45s	 = Training   runtime
	0.0s	 = Validation runtime
AutoGluon training complete, total runtime = 605.29s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_115203/")

Review AutoGluon's training run with ranking of models that did the best.

In [15]:
predictor.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
                     model   score_val  pred_time_val    fit_time  pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  fit_order
0      WeightedEnsemble_L3  -52.821924      11.568244  435.880349                0.001125           0.446518            3       True         15
1   RandomForestMSE_BAG_L2  -53.436895      10.731910  400.680140                0.594810          26.173306            2       True         12
2     ExtraTreesMSE_BAG_L2  -53.840982      10.728724  382.721570                0.591623           8.214736            2       True         14
3          LightGBM_BAG_L2  -55.049296      10.380686  401.045790                0.243585          26.538955            2       True         11
4          CatBoost_BAG_L2  -55.724578      10.220685  444.625357                0.083585          70.118522            2       True         13
5        LightGBMXT_BAG_L2  -60.146446      13.559476  427.593630                3.422376          53.086795            2       True         10
6    KNeighborsDist_BAG_L1  -84.125061       0.104031    0.030274                0.104031           0.030274            1       True          2
7      WeightedEnsemble_L2  -84.125061       0.104782    0.674625                0.000751           0.644350            2       True          9
8    KNeighborsUnif_BAG_L1 -101.546199       0.102817    0.034888                0.102817           0.034888            1       True          1
9   RandomForestMSE_BAG_L1 -116.544294       0.853333   10.812510                0.853333          10.812510            1       True          5
10    ExtraTreesMSE_BAG_L1 -124.588053       0.518733    4.847676                0.518733           4.847676            1       True          7
11         CatBoost_BAG_L1 -130.510645       0.165783  201.262563                0.165783         201.262563            1       True          6
12         LightGBM_BAG_L1 -131.054162       1.275222   27.500071                1.275222          27.500071            1       True          4
13       LightGBMXT_BAG_L1 -131.460909       6.668809   63.862681                6.668809          63.862681            1       True          3
14  NeuralNetFastAI_BAG_L1 -138.033610       0.448373   66.156172                0.448373          66.156172            1       True          8
Number of models trained: 15
Types of models trained:
{'StackerEnsembleModel_RF', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_CatBoost', 'WeightedEnsembleModel'}
Bagging used: True  (with 8 folds)
Multi-layer stack-ensembling used: True  (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('float', [])                : 3 | ['temp', 'atemp', 'windspeed']
('int', [])                  : 3 | ['season', 'weather', 'humidity']
('int', ['bool'])            : 2 | ['holiday', 'workingday']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
Plot summary of models saved to file: AutogluonModels/ag-20220521_115203/SummaryOfModels.html
*** End of fit() summary ***
Out[15]:
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
  'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
  'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
  'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
  'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
  'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
  'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
  'WeightedEnsemble_L2': 'WeightedEnsembleModel',
  'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
  'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
  'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
  'ExtraTreesMSE_BAG_L2': 'StackerEnsembleModel_XT',
  'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
 'model_performance': {'KNeighborsUnif_BAG_L1': -101.54619908446061,
  'KNeighborsDist_BAG_L1': -84.12506123181602,
  'LightGBMXT_BAG_L1': -131.46090891834504,
  'LightGBM_BAG_L1': -131.054161598899,
  'RandomForestMSE_BAG_L1': -116.54429428704391,
  'CatBoost_BAG_L1': -130.5106453332399,
  'ExtraTreesMSE_BAG_L1': -124.58805258915959,
  'NeuralNetFastAI_BAG_L1': -138.03360988203426,
  'WeightedEnsemble_L2': -84.12506123181602,
  'LightGBMXT_BAG_L2': -60.14644649980745,
  'LightGBM_BAG_L2': -55.04929612519262,
  'RandomForestMSE_BAG_L2': -53.43689487144225,
  'CatBoost_BAG_L2': -55.72457793131616,
  'ExtraTreesMSE_BAG_L2': -53.840981843999714,
  'WeightedEnsemble_L3': -52.82192379764109},
 'model_best': 'WeightedEnsemble_L3',
 'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/KNeighborsUnif_BAG_L1/',
  'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/KNeighborsDist_BAG_L1/',
  'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/LightGBMXT_BAG_L1/',
  'LightGBM_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/LightGBM_BAG_L1/',
  'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/RandomForestMSE_BAG_L1/',
  'CatBoost_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/CatBoost_BAG_L1/',
  'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/ExtraTreesMSE_BAG_L1/',
  'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/NeuralNetFastAI_BAG_L1/',
  'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_115203/models/WeightedEnsemble_L2/',
  'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/LightGBMXT_BAG_L2/',
  'LightGBM_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/LightGBM_BAG_L2/',
  'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/RandomForestMSE_BAG_L2/',
  'CatBoost_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/CatBoost_BAG_L2/',
  'ExtraTreesMSE_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/ExtraTreesMSE_BAG_L2/',
  'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_115203/models/WeightedEnsemble_L3/'},
 'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.03488755226135254,
  'KNeighborsDist_BAG_L1': 0.030274391174316406,
  'LightGBMXT_BAG_L1': 63.86268067359924,
  'LightGBM_BAG_L1': 27.500070571899414,
  'RandomForestMSE_BAG_L1': 10.81251049041748,
  'CatBoost_BAG_L1': 201.26256346702576,
  'ExtraTreesMSE_BAG_L1': 4.847675800323486,
  'NeuralNetFastAI_BAG_L1': 66.15617179870605,
  'WeightedEnsemble_L2': 0.6443502902984619,
  'LightGBMXT_BAG_L2': 53.08679533004761,
  'LightGBM_BAG_L2': 26.538954973220825,
  'RandomForestMSE_BAG_L2': 26.17330551147461,
  'CatBoost_BAG_L2': 70.11852192878723,
  'ExtraTreesMSE_BAG_L2': 8.214735507965088,
  'WeightedEnsemble_L3': 0.4465181827545166},
 'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.10281705856323242,
  'KNeighborsDist_BAG_L1': 0.10403060913085938,
  'LightGBMXT_BAG_L1': 6.668808698654175,
  'LightGBM_BAG_L1': 1.2752220630645752,
  'RandomForestMSE_BAG_L1': 0.8533329963684082,
  'CatBoost_BAG_L1': 0.16578340530395508,
  'ExtraTreesMSE_BAG_L1': 0.5187325477600098,
  'NeuralNetFastAI_BAG_L1': 0.44837307929992676,
  'WeightedEnsemble_L2': 0.0007512569427490234,
  'LightGBMXT_BAG_L2': 3.4223759174346924,
  'LightGBM_BAG_L2': 0.24358534812927246,
  'RandomForestMSE_BAG_L2': 0.5948097705841064,
  'CatBoost_BAG_L2': 0.08358502388000488,
  'ExtraTreesMSE_BAG_L2': 0.5916233062744141,
  'WeightedEnsemble_L3': 0.001125335693359375},
 'num_bag_folds': 8,
 'max_stack_level': 3,
 'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'KNeighborsDist_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'LightGBMXT_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'RandomForestMSE_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'CatBoost_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L2': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBMXT_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'RandomForestMSE_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'CatBoost_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'ExtraTreesMSE_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'WeightedEnsemble_L3': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True}},
 'leaderboard':                      model   score_val  pred_time_val    fit_time  \
 0      WeightedEnsemble_L3  -52.821924      11.568244  435.880349   
 1   RandomForestMSE_BAG_L2  -53.436895      10.731910  400.680140   
 2     ExtraTreesMSE_BAG_L2  -53.840982      10.728724  382.721570   
 3          LightGBM_BAG_L2  -55.049296      10.380686  401.045790   
 4          CatBoost_BAG_L2  -55.724578      10.220685  444.625357   
 5        LightGBMXT_BAG_L2  -60.146446      13.559476  427.593630   
 6    KNeighborsDist_BAG_L1  -84.125061       0.104031    0.030274   
 7      WeightedEnsemble_L2  -84.125061       0.104782    0.674625   
 8    KNeighborsUnif_BAG_L1 -101.546199       0.102817    0.034888   
 9   RandomForestMSE_BAG_L1 -116.544294       0.853333   10.812510   
 10    ExtraTreesMSE_BAG_L1 -124.588053       0.518733    4.847676   
 11         CatBoost_BAG_L1 -130.510645       0.165783  201.262563   
 12         LightGBM_BAG_L1 -131.054162       1.275222   27.500071   
 13       LightGBMXT_BAG_L1 -131.460909       6.668809   63.862681   
 14  NeuralNetFastAI_BAG_L1 -138.033610       0.448373   66.156172   
 
     pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  \
 0                 0.001125           0.446518            3       True   
 1                 0.594810          26.173306            2       True   
 2                 0.591623           8.214736            2       True   
 3                 0.243585          26.538955            2       True   
 4                 0.083585          70.118522            2       True   
 5                 3.422376          53.086795            2       True   
 6                 0.104031           0.030274            1       True   
 7                 0.000751           0.644350            2       True   
 8                 0.102817           0.034888            1       True   
 9                 0.853333          10.812510            1       True   
 10                0.518733           4.847676            1       True   
 11                0.165783         201.262563            1       True   
 12                1.275222          27.500071            1       True   
 13                6.668809          63.862681            1       True   
 14                0.448373          66.156172            1       True   
 
     fit_order  
 0          15  
 1          12  
 2          14  
 3          11  
 4          13  
 5          10  
 6           2  
 7           9  
 8           1  
 9           5  
 10          7  
 11          6  
 12          4  
 13          3  
 14          8  }

Create predictions from train dataset

In [16]:
from sklearn.metrics import mean_squared_error

train_pred = predictor.predict_proba(train)

train_pred_rmse = mean_squared_error(train['count'], train_pred, squared=False)
print('The train rmse is: ',train_pred_rmse)
The train rmse is:  70.86089192948288

Create predictions from test dataset

In [17]:
predictions = predictor.predict_proba(test)
predictions.head()
Out[17]:
0    23.889549
1    40.731548
2    44.783401
3    48.364571
4    51.583191
Name: count, dtype: float32

NOTE: Kaggle will reject the submission if we don't set everything to be > 0.

In [18]:
# Create submission DataFrame
submission = pd.DataFrame({'datetime': test['datetime'],
                           'count': predictions})
In [19]:
# Describe the `predictions` series to see if there are any negative values
predictions.describe()
Out[19]:
count    6493.000000
mean      100.680450
std        90.272682
min         2.973643
25%        20.340405
50%        62.326599
75%       170.001556
max       361.611725
Name: count, dtype: float64
In [20]:
# How many negative values do we have?
predictions.isna().sum()
Out[20]:
0
In [21]:
# Set them to zero
import numpy as np
submission.loc[submission['count'] < 0, 'count'] = 0
submission['count'] = submission['count'].fillna(0).astype(int)
In [22]:
# Check submissions
submission.head()
Out[22]:
datetime count
0 2011-01-20 00:00:00 23
1 2011-01-20 01:00:00 40
2 2011-01-20 02:00:00 44
3 2011-01-20 03:00:00 48
4 2011-01-20 04:00:00 51

Set predictions to submission dataframe, save, and submit

In [23]:
# create csv submission file
submission['count'] = submission['count'].astype(np.int64)
submission.to_csv("./submission.csv",index=False)
In [27]:
!kaggle competitions submit -c bike-sharing-demand -f submission.csv -m "first raw submission"
100%|█████████████████████████████████████████| 148k/148k [00:00<00:00, 237kB/s]
Successfully submitted to Bike Sharing Demand

View submission via the command line or in the web browser under the competition's page - My Submissions

In [28]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName                     date                 description                        status    publicScore  privateScore  
---------------------------  -------------------  ---------------------------------  --------  -----------  ------------  
submission.csv               2022-05-21 12:08:15  first raw submission               complete  1.82081      1.82081       
submission_new_hpo.csv       2022-05-19 10:02:34  new features with hyperparameters  complete  0.46282      0.46282       
submission_new_features.csv  2022-05-19 09:42:12  new features                       complete  0.46343      0.46343       
submission.csv               2022-05-19 09:11:03  first raw submission               complete  1.81359      1.81359       

Initial score of 1.82081

Step 4: Exploratory Data Analysis and Creating an additional feature

  • Any additional feature will do, but a great suggestion would be to separate out the datetime into hour, day, or month parts.

Data Fields

  • datetime - hourly date + timestamp
  • season - 1 = spring, 2 = summer, 3 = fall, 4 = winter
  • holiday - whether the day is considered a holiday
  • workingday - whether the day is neither a weekend nor holiday
  • weather - 1: Clear, Few clouds, Partly cloudy, Partly cloudy
  • 2: Mist + Cloudy, Mist + Broken clouds, Mist + Few clouds, Mist
  • 3: Light Snow, Light Rain + Thunderstorm + Scattered clouds, Light Rain + Scattered clouds
  • 4: Heavy Rain + Ice Pallets + Thunderstorm + Mist, Snow + Fog
  • temp - temperature in Celsius
  • atemp - "feels like" temperature in Celsius
  • humidity - relative humidity
  • windspeed - wind speed
  • casual - number of non-registered user rentals initiated
  • registered - number of registered user rentals initiated
  • count - number of total rentals
In [29]:
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib import rcParams

%matplotlib inline
In [30]:
# Create a histogram of all features to show the distribution of each one relative to the data. This is part of the exploritory data analysis
train.hist(figsize=(10, 10));
In [31]:
# plot paiplot of train
sns.pairplot(train);
In [32]:
# Plot barplot to see Renting by Weather
rcParams['figure.figsize'] = (5, 5)
sns.set_theme(style="whitegrid")

ax = sns.barplot(data=train,
            x='weather',
            y='count')
ax.set(title='Sharing by weather')
Out[32]:
[Text(0.5, 1.0, 'Sharing by weather')]

Generaly people prefer in Clear days(1) and Few clouds days(2)

In [33]:
# Plot Boxplot to find outliers in Weather
ax = sns.boxplot(x="weather", y="count", data=train)
In [34]:
# Convert datetime from objects to datetime type
train['datetime'] = pd.to_datetime(train['datetime'])
test['datetime'] = pd.to_datetime(test['datetime'])

# create new features based on datetime
train['year'] = train['datetime'].dt.year
train['month'] = train['datetime'].dt.month
train['day'] = train['datetime'].dt.day
train['hour'] = train['datetime'].dt.hour

test['year'] = test['datetime'].dt.year
test['month'] = test['datetime'].dt.month
test['day'] = test['datetime'].dt.day
test['hour'] = test['datetime'].dt.hour
In [35]:
# Plot the distribution of renting by Hour
rcParams['figure.figsize'] = (15, 8)
fig,ax = plt.subplots()
sns.pointplot(data=train[['hour', 'count', 'season']],
              x='hour',
              y='count',
              hue='season',
              ax=ax)
ax.set(title="Season hourly distribution of sharing")
Out[35]:
[Text(0.5, 1.0, 'Season hourly distribution of sharing')]
In [36]:
# Drop datetime
train.drop(train[['datetime']], axis=1, inplace=True)
test.drop(test[['datetime']], axis=1, inplace=True)
In [37]:
# Check correlations after add more features
rcParams['figure.figsize'] = (12, 10)
ax = sns.heatmap(
    train.corr(),
    cmap="YlGnBu",
    square=True,
    annot=True
)

Make category types for these so models know they are not just numbers

  • AutoGluon originally sees these as ints, but in reality they are int representations of a category.
  • Setting the dtype to category will classify these as categories in AutoGluon.
In [38]:
# convert to categorical
train["season"] = pd.Categorical(train["season"])
train["weather"] = pd.Categorical(train["weather"])
train["holiday"] = pd.Categorical(train["holiday"])
train["workingday"] = pd.Categorical(train["workingday"])

test["season"] = pd.Categorical(test["season"])
test["weather"] = pd.Categorical(test["weather"])
test["holiday"] = pd.Categorical(test["holiday"])
test["workingday"] = pd.Categorical(test["workingday"])
In [39]:
# View are new feature
train.head()
Out[39]:
season holiday workingday weather temp atemp humidity windspeed casual registered count year month day hour
0 1 0 0 1 9.84 14.395 81 0.0 3 13 16 2011 1 1 0
1 1 0 0 1 9.02 13.635 80 0.0 8 32 40 2011 1 1 1
2 1 0 0 1 9.02 13.635 80 0.0 5 27 32 2011 1 1 2
3 1 0 0 1 9.84 14.395 75 0.0 3 10 13 2011 1 1 3
4 1 0 0 1 9.84 14.395 75 0.0 0 1 1 2011 1 1 4
In [40]:
# View histogram of all features again now with the hour feature
train.hist(figsize=(10, 10));

Step 5: Rerun the model with the same settings as before, just with more features

In [41]:
predictor_new_features = TabularPredictor(
    label='count',
    problem_type='regression',
    eval_metric='root_mean_squared_error',
    learner_kwargs={"ignored_columns": ["casual", "registered"]}
).fit(
    train_data=train,
    time_limit=900,
    presets='best_quality'
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_121113/"
Presets specified: ['best_quality']
Beginning AutoGluon training ... Time limit = 900s
AutoGluon will save models to "AutogluonModels/ag-20220521_121113/"
AutoGluon Version:  0.4.1
Python Version:     3.7.10
Operating System:   Linux
Train Data Rows:    10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
	Available Memory:                    1794.16 MB
	Train Data (Original)  Memory Usage: 0.74 MB (0.0% of available memory)
	Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
	Stage 1 Generators:
		Fitting AsTypeFeatureGenerator...
			Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
	Stage 2 Generators:
		Fitting FillNaFeatureGenerator...
	Stage 3 Generators:
		Fitting IdentityFeatureGenerator...
		Fitting CategoryFeatureGenerator...
			Fitting CategoryMemoryMinimizeFeatureGenerator...
	Stage 4 Generators:
		Fitting DropUniqueFeatureGenerator...
	Types of features in original data (raw dtype, special dtypes):
		('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
		('float', [])    : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])      : 5 | ['humidity', 'year', 'month', 'day', 'hour']
	Types of features in processed data (raw dtype, special dtypes):
		('category', [])  : 2 | ['season', 'weather']
		('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
		('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
	0.1s = Fit runtime
	12 features in original data used to generate 12 features in processed data.
	Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.19s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
	To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 599.72s of the 899.79s of remaining time.
	-123.9333	 = Validation score   (root_mean_squared_error)
	0.03s	 = Training   runtime
	0.2s	 = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 599.24s of the 899.31s of remaining time.
	-119.3656	 = Validation score   (root_mean_squared_error)
	0.03s	 = Training   runtime
	0.21s	 = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 598.76s of the 898.83s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-37.0978	 = Validation score   (root_mean_squared_error)
	93.72s	 = Training   runtime
	17.59s	 = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 499.28s of the 799.35s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-37.6929	 = Validation score   (root_mean_squared_error)
	44.41s	 = Training   runtime
	3.65s	 = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 450.32s of the 750.4s of remaining time.
	-42.1495	 = Validation score   (root_mean_squared_error)
	9.36s	 = Training   runtime
	0.57s	 = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 437.79s of the 737.86s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-37.8729	 = Validation score   (root_mean_squared_error)
	360.07s	 = Training   runtime
	0.34s	 = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 74.82s of the 374.89s of remaining time.
	-41.6234	 = Validation score   (root_mean_squared_error)
	5.78s	 = Training   runtime
	0.55s	 = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 65.9s of the 365.98s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-55.7648	 = Validation score   (root_mean_squared_error)
	73.39s	 = Training   runtime
	0.53s	 = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 289.56s of remaining time.
	-35.8693	 = Validation score   (root_mean_squared_error)
	1.03s	 = Training   runtime
	0.0s	 = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 288.44s of the 288.42s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-36.9157	 = Validation score   (root_mean_squared_error)
	20.56s	 = Training   runtime
	0.29s	 = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 265.26s of the 265.24s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-36.1403	 = Validation score   (root_mean_squared_error)
	20.15s	 = Training   runtime
	0.15s	 = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 242.44s of the 242.41s of remaining time.
	-36.661	 = Validation score   (root_mean_squared_error)
	26.85s	 = Training   runtime
	0.61s	 = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 212.57s of the 212.55s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-36.1668	 = Validation score   (root_mean_squared_error)
	54.01s	 = Training   runtime
	0.12s	 = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L2 ... Training model for up to 155.9s of the 155.88s of remaining time.
	-36.1166	 = Validation score   (root_mean_squared_error)
	7.95s	 = Training   runtime
	0.59s	 = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L2 ... Training model for up to 144.8s of the 144.78s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-36.318	 = Validation score   (root_mean_squared_error)
	107.9s	 = Training   runtime
	0.54s	 = Validation runtime
Fitting model: XGBoost_BAG_L2 ... Training model for up to 33.88s of the 33.86s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	-36.5113	 = Validation score   (root_mean_squared_error)
	23.33s	 = Training   runtime
	0.15s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2 ... Training model for up to 7.48s of the 7.46s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
	Warning: Exception caused NeuralNetTorch_BAG_L2 to fail during training... Skipping this model.
		ray::_ray_fit() (pid=4801, ip=169.255.254.2)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 289, in _ray_fit
    time_limit=time_limit_fold, num_cpus=num_cpus, **kwargs_fold)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
    out = self._fit(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 203, in _fit
    **fit_kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 377, in _train_net
    raise AssertionError('0 epochs trained!')
AssertionError: 0 epochs trained!
Detailed Traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/trainer/abstract_trainer.py", line 1074, in _train_and_save
    model = self._train_single(X, y, model, X_val, y_val, **model_fit_kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/trainer/abstract_trainer.py", line 1032, in _train_single
    model = model.fit(X=X, y=y, X_val=X_val, y_val=y_val, **model_fit_kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
    out = self._fit(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 153, in _fit
    return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 234, in _fit
    n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 502, in _fit_folds
    fold_fitting_strategy.after_all_folds_scheduled()
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 455, in after_all_folds_scheduled
    raise processed_exception
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 423, in after_all_folds_scheduled
    time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
  File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/ray/worker.py", line 1733, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AssertionError): ray::_ray_fit() (pid=4801, ip=169.255.254.2)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 289, in _ray_fit
    time_limit=time_limit_fold, num_cpus=num_cpus, **kwargs_fold)
  File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
    out = self._fit(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 203, in _fit
    **fit_kwargs)
  File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 377, in _train_net
    raise AssertionError('0 epochs trained!')
AssertionError: 0 epochs trained!
Fitting model: LightGBMLarge_BAG_L2 ... Training model for up to 0.38s of the 0.36s of remaining time.
	Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 12:26:14,257	WARNING services.py:1866 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=0.73gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.
	Time limit exceeded... Skipping LightGBMLarge_BAG_L2.
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -3.66s of remaining time.
	-35.7039	 = Validation score   (root_mean_squared_error)
	0.49s	 = Training   runtime
	0.0s	 = Validation runtime
AutoGluon training complete, total runtime = 904.38s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_121113/")
In [42]:
predictor_new_features.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
                     model   score_val  pred_time_val    fit_time  pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  fit_order
0      WeightedEnsemble_L3  -35.703944      25.801695  827.468261                0.001348           0.487528            3       True         17
1      WeightedEnsemble_L2  -35.869295      22.151278  508.585230                0.000794           1.028767            2       True          9
2     ExtraTreesMSE_BAG_L2  -36.116601      24.237407  594.732446                0.592123           7.949546            2       True         14
3          LightGBM_BAG_L2  -36.140333      23.796642  606.932092                0.151358          20.149193            2       True         11
4          CatBoost_BAG_L2  -36.166801      23.760818  640.795617                0.115534          54.012717            2       True         13
5   NeuralNetFastAI_BAG_L2  -36.318005      24.180289  694.683991                0.535005         107.901092            2       True         15
6           XGBoost_BAG_L2  -36.511258      23.794355  610.117719                0.149071          23.334819            2       True         16
7   RandomForestMSE_BAG_L2  -36.661025      24.257257  613.633366                0.611973          26.850466            2       True         12
8        LightGBMXT_BAG_L2  -36.915742      23.938235  607.344602                0.292951          20.561702            2       True         10
9        LightGBMXT_BAG_L1  -37.097828      17.588601   93.720337               17.588601          93.720337            1       True          3
10         LightGBM_BAG_L1  -37.692850       3.653144   44.405111                3.653144          44.405111            1       True          4
11         CatBoost_BAG_L1  -37.872933       0.342373  360.073151                0.342373         360.073151            1       True          6
12    ExtraTreesMSE_BAG_L1  -41.623410       0.550196    5.776536                0.550196           5.776536            1       True          7
13  RandomForestMSE_BAG_L1  -42.149545       0.566365    9.357864                0.566365           9.357864            1       True          5
14  NeuralNetFastAI_BAG_L1  -55.764827       0.534983   73.393159                0.534983          73.393159            1       True          8
15   KNeighborsDist_BAG_L1 -119.365601       0.206364    0.027186                0.206364           0.027186            1       True          2
16   KNeighborsUnif_BAG_L1 -123.933260       0.203257    0.029556                0.203257           0.029556            1       True          1
Number of models trained: 17
Types of models trained:
{'StackerEnsembleModel_RF', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_XGBoost', 'StackerEnsembleModel_CatBoost', 'WeightedEnsembleModel'}
Bagging used: True  (with 8 folds)
Multi-layer stack-ensembling used: True  (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', [])  : 2 | ['season', 'weather']
('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_121113/SummaryOfModels.html
*** End of fit() summary ***
Out[42]:
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
  'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
  'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
  'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
  'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
  'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
  'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
  'WeightedEnsemble_L2': 'WeightedEnsembleModel',
  'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
  'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
  'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
  'ExtraTreesMSE_BAG_L2': 'StackerEnsembleModel_XT',
  'NeuralNetFastAI_BAG_L2': 'StackerEnsembleModel_NNFastAiTabular',
  'XGBoost_BAG_L2': 'StackerEnsembleModel_XGBoost',
  'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
 'model_performance': {'KNeighborsUnif_BAG_L1': -123.93326033133259,
  'KNeighborsDist_BAG_L1': -119.36560130583281,
  'LightGBMXT_BAG_L1': -37.09782821469706,
  'LightGBM_BAG_L1': -37.69285008347106,
  'RandomForestMSE_BAG_L1': -42.14954493458521,
  'CatBoost_BAG_L1': -37.87293345815347,
  'ExtraTreesMSE_BAG_L1': -41.623410397279095,
  'NeuralNetFastAI_BAG_L1': -55.764827348159514,
  'WeightedEnsemble_L2': -35.86929496695115,
  'LightGBMXT_BAG_L2': -36.915742419191254,
  'LightGBM_BAG_L2': -36.14033348245729,
  'RandomForestMSE_BAG_L2': -36.66102537394624,
  'CatBoost_BAG_L2': -36.16680134531193,
  'ExtraTreesMSE_BAG_L2': -36.11660135289378,
  'NeuralNetFastAI_BAG_L2': -36.31800497190385,
  'XGBoost_BAG_L2': -36.51125766145703,
  'WeightedEnsemble_L3': -35.70394398419218},
 'model_best': 'WeightedEnsemble_L3',
 'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/KNeighborsUnif_BAG_L1/',
  'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/KNeighborsDist_BAG_L1/',
  'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/LightGBMXT_BAG_L1/',
  'LightGBM_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/LightGBM_BAG_L1/',
  'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/RandomForestMSE_BAG_L1/',
  'CatBoost_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/CatBoost_BAG_L1/',
  'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/ExtraTreesMSE_BAG_L1/',
  'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/NeuralNetFastAI_BAG_L1/',
  'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_121113/models/WeightedEnsemble_L2/',
  'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/LightGBMXT_BAG_L2/',
  'LightGBM_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/LightGBM_BAG_L2/',
  'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/RandomForestMSE_BAG_L2/',
  'CatBoost_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/CatBoost_BAG_L2/',
  'ExtraTreesMSE_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/ExtraTreesMSE_BAG_L2/',
  'NeuralNetFastAI_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/NeuralNetFastAI_BAG_L2/',
  'XGBoost_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/XGBoost_BAG_L2/',
  'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_121113/models/WeightedEnsemble_L3/'},
 'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.029555797576904297,
  'KNeighborsDist_BAG_L1': 0.02718639373779297,
  'LightGBMXT_BAG_L1': 93.72033739089966,
  'LightGBM_BAG_L1': 44.40511083602905,
  'RandomForestMSE_BAG_L1': 9.357863903045654,
  'CatBoost_BAG_L1': 360.0731511116028,
  'ExtraTreesMSE_BAG_L1': 5.776535511016846,
  'NeuralNetFastAI_BAG_L1': 73.39315891265869,
  'WeightedEnsemble_L2': 1.0287666320800781,
  'LightGBMXT_BAG_L2': 20.561702489852905,
  'LightGBM_BAG_L2': 20.149192571640015,
  'RandomForestMSE_BAG_L2': 26.85046625137329,
  'CatBoost_BAG_L2': 54.01271724700928,
  'ExtraTreesMSE_BAG_L2': 7.9495463371276855,
  'NeuralNetFastAI_BAG_L2': 107.90109157562256,
  'XGBoost_BAG_L2': 23.334819078445435,
  'WeightedEnsemble_L3': 0.48752784729003906},
 'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.20325732231140137,
  'KNeighborsDist_BAG_L1': 0.20636367797851562,
  'LightGBMXT_BAG_L1': 17.5886013507843,
  'LightGBM_BAG_L1': 3.653143882751465,
  'RandomForestMSE_BAG_L1': 0.5663654804229736,
  'CatBoost_BAG_L1': 0.3423731327056885,
  'ExtraTreesMSE_BAG_L1': 0.5501961708068848,
  'NeuralNetFastAI_BAG_L1': 0.5349831581115723,
  'WeightedEnsemble_L2': 0.0007936954498291016,
  'LightGBMXT_BAG_L2': 0.2929508686065674,
  'LightGBM_BAG_L2': 0.15135788917541504,
  'RandomForestMSE_BAG_L2': 0.6119728088378906,
  'CatBoost_BAG_L2': 0.11553406715393066,
  'ExtraTreesMSE_BAG_L2': 0.5921225547790527,
  'NeuralNetFastAI_BAG_L2': 0.5350046157836914,
  'XGBoost_BAG_L2': 0.14907097816467285,
  'WeightedEnsemble_L3': 0.001348257064819336},
 'num_bag_folds': 8,
 'max_stack_level': 3,
 'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'KNeighborsDist_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'LightGBMXT_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'RandomForestMSE_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'CatBoost_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L2': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBMXT_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'RandomForestMSE_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'CatBoost_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'ExtraTreesMSE_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True,
   'use_child_oof': True},
  'NeuralNetFastAI_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'XGBoost_BAG_L2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L3': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True}},
 'leaderboard':                      model   score_val  pred_time_val    fit_time  \
 0      WeightedEnsemble_L3  -35.703944      25.801695  827.468261   
 1      WeightedEnsemble_L2  -35.869295      22.151278  508.585230   
 2     ExtraTreesMSE_BAG_L2  -36.116601      24.237407  594.732446   
 3          LightGBM_BAG_L2  -36.140333      23.796642  606.932092   
 4          CatBoost_BAG_L2  -36.166801      23.760818  640.795617   
 5   NeuralNetFastAI_BAG_L2  -36.318005      24.180289  694.683991   
 6           XGBoost_BAG_L2  -36.511258      23.794355  610.117719   
 7   RandomForestMSE_BAG_L2  -36.661025      24.257257  613.633366   
 8        LightGBMXT_BAG_L2  -36.915742      23.938235  607.344602   
 9        LightGBMXT_BAG_L1  -37.097828      17.588601   93.720337   
 10         LightGBM_BAG_L1  -37.692850       3.653144   44.405111   
 11         CatBoost_BAG_L1  -37.872933       0.342373  360.073151   
 12    ExtraTreesMSE_BAG_L1  -41.623410       0.550196    5.776536   
 13  RandomForestMSE_BAG_L1  -42.149545       0.566365    9.357864   
 14  NeuralNetFastAI_BAG_L1  -55.764827       0.534983   73.393159   
 15   KNeighborsDist_BAG_L1 -119.365601       0.206364    0.027186   
 16   KNeighborsUnif_BAG_L1 -123.933260       0.203257    0.029556   
 
     pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  \
 0                 0.001348           0.487528            3       True   
 1                 0.000794           1.028767            2       True   
 2                 0.592123           7.949546            2       True   
 3                 0.151358          20.149193            2       True   
 4                 0.115534          54.012717            2       True   
 5                 0.535005         107.901092            2       True   
 6                 0.149071          23.334819            2       True   
 7                 0.611973          26.850466            2       True   
 8                 0.292951          20.561702            2       True   
 9                17.588601          93.720337            1       True   
 10                3.653144          44.405111            1       True   
 11                0.342373         360.073151            1       True   
 12                0.550196           5.776536            1       True   
 13                0.566365           9.357864            1       True   
 14                0.534983          73.393159            1       True   
 15                0.206364           0.027186            1       True   
 16                0.203257           0.029556            1       True   
 
     fit_order  
 0          17  
 1           9  
 2          14  
 3          11  
 4          13  
 5          15  
 6          16  
 7          12  
 8          10  
 9           3  
 10          4  
 11          6  
 12          7  
 13          5  
 14          8  
 15          2  
 16          1  }

Create predictions for train dataset

In [43]:
# predictor_new_features for training
predictions_new_features_train = predictor_new_features.predict_proba(train)

rmse_new_features = mean_squared_error(train['count'], predictions_new_features_train, squared=False)
print('The rmse for new feateares in train is: ',rmse_new_features)
The rmse for new feateares in train is:  18.306591272491698

Create predictions for test dataset

In [44]:
#predictor_new_features for testing
predictions_new_features = predictor_new_features.predict_proba(test)
In [45]:
# Remember to set all negative values to zero
submission_new_features = pd.DataFrame({'datetime': submission['datetime'],
                           'count': predictions_new_features})

submission_new_features.loc[submission_new_features['count'] <0, 'count'] = 0
In [46]:
# Same submitting predictions
submission_new_features['count'] = submission_new_features['count'].fillna(0).astype(int)
submission_new_features['count'] = submission_new_features['count'].astype(np.int64)

submission_new_features.to_csv("submission_new_features.csv", index=False)
In [47]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_features.csv -m "new features"
100%|█████████████████████████████████████████| 149k/149k [00:00<00:00, 256kB/s]
Successfully submitted to Bike Sharing Demand
In [48]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName                     date                 description                        status    publicScore  privateScore  
---------------------------  -------------------  ---------------------------------  --------  -----------  ------------  
submission_new_features.csv  2022-05-21 12:28:51  new features                       complete  0.46603      0.46603       
submission.csv               2022-05-21 12:08:15  first raw submission               complete  1.82081      1.82081       
submission_new_hpo.csv       2022-05-19 10:02:34  new features with hyperparameters  complete  0.46282      0.46282       
submission_new_features.csv  2022-05-19 09:42:12  new features                       complete  0.46343      0.46343       

New Score of 0.46603

Step 6: Hyper parameter optimization

  • There are many options for hyper parameter optimization.
  • Options are to change the AutoGluon higher level parameters or the individual model hyperparameters.
  • The hyperparameters of the models themselves that are in AutoGluon. Those need the hyperparameter and hyperparameter_tune_kwargs arguments.
In [51]:
# Defining hyperparameter and hyperparameter_tune_kwargs
import autogluon.core as ag

nn_options = {
    'num_epochs': 10,
    'learning_rate': ag.space.Real(1e-4, 1e-2, default=5e-4, log=True),
    'activation': ag.space.Categorical('relu', 'softrelu', 'tanh'),
    'dropout_prob': ag.space.Real(0.0, 0.5, default=0.1),
}

gbm_options = { 
    'num_boost_round': 100,
    'num_leaves': ag.space.Int(lower=26, upper=66, default=36),
}


hyperparameters = {
    'GBM': gbm_options,
    'NN_TORCH': nn_options,
}

hyperparameter_tune_kwargs = { 
    'num_trials': 5,
    'scheduler' : 'local',
    'searcher': 'auto',
}
In [52]:
predictor_new_hpo = TabularPredictor(
    label="count",
    eval_metric='root_mean_squared_error',
    learner_kwargs={'ignored_columns': ["casual", "registered"]}
).fit(
    train_data=train,
    time_limit=1200,
    num_bag_folds=5,
    num_bag_sets=1,
    num_stack_levels=1,
    presets="best_quality",
    hyperparameters=hyperparameters,
    hyperparameter_tune_kwargs=hyperparameter_tune_kwargs,
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_123126/"
Presets specified: ['best_quality']
Warning: hyperparameter tuning is currently experimental and may cause the process to hang.
Beginning AutoGluon training ... Time limit = 1200s
AutoGluon will save models to "AutogluonModels/ag-20220521_123126/"
AutoGluon Version:  0.4.1
Python Version:     3.7.10
Operating System:   Linux
Train Data Rows:    10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
AutoGluon infers your prediction problem is: 'regression' (because dtype of label-column == int and many unique label-values observed).
	Label info (max, min, mean, stddev): (977, 1, 191.57413, 181.14445)
	If 'regression' is not the correct problem_type, please manually specify the problem_type parameter during predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression'])
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
	Available Memory:                    1873.43 MB
	Train Data (Original)  Memory Usage: 0.74 MB (0.0% of available memory)
	Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
	Stage 1 Generators:
		Fitting AsTypeFeatureGenerator...
			Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
	Stage 2 Generators:
		Fitting FillNaFeatureGenerator...
	Stage 3 Generators:
		Fitting IdentityFeatureGenerator...
		Fitting CategoryFeatureGenerator...
			Fitting CategoryMemoryMinimizeFeatureGenerator...
	Stage 4 Generators:
		Fitting DropUniqueFeatureGenerator...
	Types of features in original data (raw dtype, special dtypes):
		('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
		('float', [])    : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])      : 5 | ['humidity', 'year', 'month', 'day', 'hour']
	Types of features in processed data (raw dtype, special dtypes):
		('category', [])  : 2 | ['season', 'weather']
		('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
		('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
	0.1s = Fit runtime
	12 features in original data used to generate 12 features in processed data.
	Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.18s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
	To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 2 L1 models ...
Hyperparameter tuning model: LightGBM_BAG_L1 ... Tuning model for up to 71.97s of the 1199.82s of remaining time.
100%|██████████| 5/5 [00:02<00:00,  2.37it/s]
Fitted model: LightGBM_BAG_L1/T1 ...
	-45.1728	 = Validation score   (root_mean_squared_error)
	0.38s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T2 ...
	-42.4191	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T3 ...
	-44.8062	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T4 ...
	-126.8104	 = Validation score   (root_mean_squared_error)
	0.3s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T5 ...
	-48.5942	 = Validation score   (root_mean_squared_error)
	0.34s	 = Training   runtime
	0.02s	 = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L1 ... Tuning model for up to 71.97s of the 1196.89s of remaining time.
 80%|████████  | 4/5 [00:37<00:10, 10.73s/it]	Stopping HPO to satisfy time limit...
 80%|████████  | 4/5 [00:56<00:14, 14.16s/it]
Fitted model: NeuralNetTorch_BAG_L1/T1 ...
	-102.1329	 = Validation score   (root_mean_squared_error)
	5.42s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T2 ...
	-58.9307	 = Validation score   (root_mean_squared_error)
	8.91s	 = Training   runtime
	0.05s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T3 ...
	-84.9915	 = Validation score   (root_mean_squared_error)
	8.0s	 = Training   runtime
	0.05s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T4 ...
	-56.5326	 = Validation score   (root_mean_squared_error)
	14.93s	 = Training   runtime
	0.06s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T5 ...
	-145.217	 = Validation score   (root_mean_squared_error)
	18.51s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 738.55s of the 1138.69s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-44.719	 = Validation score   (root_mean_squared_error)
	7.46s	 = Training   runtime
	0.11s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 729.19s of the 1129.32s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-42.8473	 = Validation score   (root_mean_squared_error)
	6.55s	 = Training   runtime
	0.13s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 720.36s of the 1120.49s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-43.6524	 = Validation score   (root_mean_squared_error)
	6.9s	 = Training   runtime
	0.14s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T4 ... Training model for up to 711.2s of the 1111.34s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-124.3498	 = Validation score   (root_mean_squared_error)
	6.9s	 = Training   runtime
	0.09s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T5 ... Training model for up to 701.92s of the 1102.05s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 12:33:06,004	WARNING worker.py:1257 -- The actor or task with ID 275e16fd1756370196a0a6166725870344cbe5fe01000000 cannot be scheduled right now. You can ignore this message if this Ray cluster is expected to auto-scale or if you specified a runtime_env for this actor or task, which may take time to install.  Otherwise, this is likely due to all cluster resources being claimed by actors. To resolve the issue, consider creating fewer actors or increasing the resources available to this Ray cluster.
Required resources for this actor or task: {CPU: 1.000000}
Available resources on this node: {1.000000/2.000000 CPU, 1.319749 GiB/1.319749 GiB memory, 0.659874 GiB/0.659874 GiB object_store_memory, 1.000000/1.000000 node:169.255.254.2}
 In total there are 2 pending tasks and 0 pending actors on this node.
	-47.8441	 = Validation score   (root_mean_squared_error)
	7.73s	 = Training   runtime
	0.12s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T1 ... Training model for up to 691.07s of the 1091.2s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-96.49	 = Validation score   (root_mean_squared_error)
	25.51s	 = Training   runtime
	0.39s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T2 ... Training model for up to 667.71s of the 1067.85s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-56.923	 = Validation score   (root_mean_squared_error)
	38.11s	 = Training   runtime
	0.31s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T3 ... Training model for up to 635.55s of the 1035.68s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-81.996	 = Validation score   (root_mean_squared_error)
	35.47s	 = Training   runtime
	0.34s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T4 ... Training model for up to 604.4s of the 1004.54s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-53.8145	 = Validation score   (root_mean_squared_error)
	72.15s	 = Training   runtime
	0.43s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T5 ... Training model for up to 544.51s of the 944.64s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-140.6635	 = Validation score   (root_mean_squared_error)
	88.63s	 = Training   runtime
	0.49s	 = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 871.12s of remaining time.
	-42.2555	 = Validation score   (root_mean_squared_error)
	0.8s	 = Training   runtime
	0.0s	 = Validation runtime
Fitting 2 L2 models ...
Hyperparameter tuning model: LightGBM_BAG_L2 ... Tuning model for up to 78.32s of the 870.22s of remaining time.
100%|██████████| 5/5 [00:02<00:00,  1.80it/s]
Fitted model: LightGBM_BAG_L2/T1 ...
	-43.6231	 = Validation score   (root_mean_squared_error)
	0.46s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T2 ...
	-43.3439	 = Validation score   (root_mean_squared_error)
	0.42s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T3 ...
	-43.7476	 = Validation score   (root_mean_squared_error)
	0.58s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T4 ...
	-105.803	 = Validation score   (root_mean_squared_error)
	0.43s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T5 ...
	-43.9841	 = Validation score   (root_mean_squared_error)
	0.51s	 = Training   runtime
	0.01s	 = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L2 ... Tuning model for up to 78.32s of the 866.66s of remaining time.
100%|██████████| 5/5 [00:48<00:00,  9.75s/it]
Fitted model: NeuralNetTorch_BAG_L2/T1 ...
	-45.5715	 = Validation score   (root_mean_squared_error)
	5.24s	 = Training   runtime
	0.05s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T2 ...
	-45.7959	 = Validation score   (root_mean_squared_error)
	7.09s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T3 ...
	-48.7631	 = Validation score   (root_mean_squared_error)
	6.34s	 = Training   runtime
	0.06s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T4 ...
	-44.9841	 = Validation score   (root_mean_squared_error)
	12.4s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T5 ...
	-47.2174	 = Validation score   (root_mean_squared_error)
	16.73s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T1 ... Training model for up to 816.42s of the 816.4s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-41.2004	 = Validation score   (root_mean_squared_error)
	7.92s	 = Training   runtime
	0.1s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T2 ... Training model for up to 805.65s of the 805.63s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-41.0374	 = Validation score   (root_mean_squared_error)
	7.11s	 = Training   runtime
	0.1s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T3 ... Training model for up to 796.0s of the 795.98s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-41.3462	 = Validation score   (root_mean_squared_error)
	7.99s	 = Training   runtime
	0.14s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T4 ... Training model for up to 784.58s of the 784.55s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-104.121	 = Validation score   (root_mean_squared_error)
	7.18s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T5 ... Training model for up to 775.08s of the 775.05s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-41.6475	 = Validation score   (root_mean_squared_error)
	7.91s	 = Training   runtime
	0.11s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T1 ... Training model for up to 764.83s of the 764.81s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-42.5771	 = Validation score   (root_mean_squared_error)
	25.69s	 = Training   runtime
	0.33s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T2 ... Training model for up to 740.9s of the 740.88s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-42.6448	 = Validation score   (root_mean_squared_error)
	36.17s	 = Training   runtime
	0.34s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T3 ... Training model for up to 708.82s of the 708.8s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-45.6104	 = Validation score   (root_mean_squared_error)
	35.41s	 = Training   runtime
	0.42s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T4 ... Training model for up to 676.28s of the 676.26s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-42.5829	 = Validation score   (root_mean_squared_error)
	60.85s	 = Training   runtime
	0.6s	 = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T5 ... Training model for up to 624.87s of the 624.85s of remaining time.
	Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
	-43.8804	 = Validation score   (root_mean_squared_error)
	81.01s	 = Training   runtime
	0.72s	 = Validation runtime
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the 556.88s of remaining time.
	-40.7067	 = Validation score   (root_mean_squared_error)
	0.87s	 = Training   runtime
	0.0s	 = Validation runtime
AutoGluon training complete, total runtime = 644.24s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_123126/")
In [53]:
predictor_new_hpo.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
                       model   score_val  pred_time_val    fit_time  pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  fit_order
0        WeightedEnsemble_L3  -40.706686       5.304729  558.431824                0.001316           0.874229            3       True         22
1         LightGBM_BAG_L2/T2  -41.037416       2.635418  302.515243                0.095234           7.105219            2       True         13
2         LightGBM_BAG_L2/T1  -41.200410       2.642720  303.331255                0.102536           7.921231            2       True         12
3         LightGBM_BAG_L2/T3  -41.346167       2.680607  303.401063                0.140424           7.991039            2       True         14
4         LightGBM_BAG_L2/T5  -41.647512       2.650338  303.322842                0.110154           7.912818            2       True         16
5        WeightedEnsemble_L2  -42.255533       1.008055  124.509622                0.000832           0.797046            2       True         11
6   NeuralNetTorch_BAG_L2/T1  -42.577094       2.868365  321.103085                0.328181          25.693061            2       True         17
7   NeuralNetTorch_BAG_L2/T4  -42.582856       3.144919  356.262383                0.604735          60.852359            2       True         20
8   NeuralNetTorch_BAG_L2/T2  -42.644782       2.884022  331.575251                0.343839          36.165227            2       True         18
9         LightGBM_BAG_L1/T2  -42.847297       0.129618    6.548905                0.129618           6.548905            1       True          2
10        LightGBM_BAG_L1/T3  -43.652435       0.135641    6.900732                0.135641           6.900732            1       True          3
11  NeuralNetTorch_BAG_L2/T5  -43.880374       3.265158  376.423512                0.724974          81.013488            2       True         21
12        LightGBM_BAG_L1/T1  -44.719007       0.110728    7.457866                0.110728           7.457866            1       True          1
13  NeuralNetTorch_BAG_L2/T3  -45.610407       2.963489  330.815973                0.423306          35.405949            2       True         19
14        LightGBM_BAG_L1/T5  -47.844107       0.117313    7.732576                0.117313           7.732576            1       True          5
15  NeuralNetTorch_BAG_L1/T4  -53.814540       0.427469   72.154497                0.427469          72.154497            1       True          9
16  NeuralNetTorch_BAG_L1/T2  -56.923024       0.314495   38.108442                0.314495          38.108442            1       True          7
17  NeuralNetTorch_BAG_L1/T3  -81.995966       0.336665   35.471487                0.336665          35.471487            1       True          8
18  NeuralNetTorch_BAG_L1/T1  -96.490001       0.385836   25.508792                0.385836          25.508792            1       True          6
19        LightGBM_BAG_L2/T4 -104.120960       2.621153  302.590129                0.080969           7.180105            2       True         15
20        LightGBM_BAG_L1/T4 -124.349788       0.094529    6.897573                0.094529           6.897573            1       True          4
21  NeuralNetTorch_BAG_L1/T5 -140.663465       0.487890   88.629153                0.487890          88.629153            1       True         10
Number of models trained: 22
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_TabularNeuralNetTorch'}
Bagging used: True  (with 5 folds)
Multi-layer stack-ensembling used: True  (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', [])  : 2 | ['season', 'weather']
('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_123126/SummaryOfModels.html
*** End of fit() summary ***
Out[53]:
{'model_types': {'LightGBM_BAG_L1/T1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T3': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T4': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T5': 'StackerEnsembleModel_LGB',
  'NeuralNetTorch_BAG_L1/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'WeightedEnsemble_L2': 'WeightedEnsembleModel',
  'LightGBM_BAG_L2/T1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T3': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T4': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T5': 'StackerEnsembleModel_LGB',
  'NeuralNetTorch_BAG_L2/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
 'model_performance': {'LightGBM_BAG_L1/T1': -44.719006568986714,
  'LightGBM_BAG_L1/T2': -42.84729671304735,
  'LightGBM_BAG_L1/T3': -43.65243479420564,
  'LightGBM_BAG_L1/T4': -124.34978848933305,
  'LightGBM_BAG_L1/T5': -47.844107439841125,
  'NeuralNetTorch_BAG_L1/T1': -96.49000098962254,
  'NeuralNetTorch_BAG_L1/T2': -56.923023716520504,
  'NeuralNetTorch_BAG_L1/T3': -81.995966464309,
  'NeuralNetTorch_BAG_L1/T4': -53.814540417474646,
  'NeuralNetTorch_BAG_L1/T5': -140.66346547927142,
  'WeightedEnsemble_L2': -42.255532640311394,
  'LightGBM_BAG_L2/T1': -41.20040982056577,
  'LightGBM_BAG_L2/T2': -41.03741568144702,
  'LightGBM_BAG_L2/T3': -41.34616693698568,
  'LightGBM_BAG_L2/T4': -104.12096022803408,
  'LightGBM_BAG_L2/T5': -41.64751175358479,
  'NeuralNetTorch_BAG_L2/T1': -42.577093629311705,
  'NeuralNetTorch_BAG_L2/T2': -42.64478220404302,
  'NeuralNetTorch_BAG_L2/T3': -45.61040725200952,
  'NeuralNetTorch_BAG_L2/T4': -42.58285563687108,
  'NeuralNetTorch_BAG_L2/T5': -43.88037422572748,
  'WeightedEnsemble_L3': -40.70668617525216},
 'model_best': 'WeightedEnsemble_L3',
 'model_paths': {'LightGBM_BAG_L1/T1': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T1/',
  'LightGBM_BAG_L1/T2': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T2/',
  'LightGBM_BAG_L1/T3': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T3/',
  'LightGBM_BAG_L1/T4': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T4/',
  'LightGBM_BAG_L1/T5': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T5/',
  'NeuralNetTorch_BAG_L1/T1': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T1/',
  'NeuralNetTorch_BAG_L1/T2': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T2/',
  'NeuralNetTorch_BAG_L1/T3': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T3/',
  'NeuralNetTorch_BAG_L1/T4': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T4/',
  'NeuralNetTorch_BAG_L1/T5': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T5/',
  'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_123126/models/WeightedEnsemble_L2/',
  'LightGBM_BAG_L2/T1': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T1/',
  'LightGBM_BAG_L2/T2': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T2/',
  'LightGBM_BAG_L2/T3': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T3/',
  'LightGBM_BAG_L2/T4': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T4/',
  'LightGBM_BAG_L2/T5': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T5/',
  'NeuralNetTorch_BAG_L2/T1': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T1/',
  'NeuralNetTorch_BAG_L2/T2': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T2/',
  'NeuralNetTorch_BAG_L2/T3': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T3/',
  'NeuralNetTorch_BAG_L2/T4': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T4/',
  'NeuralNetTorch_BAG_L2/T5': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T5/',
  'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_123126/models/WeightedEnsemble_L3/'},
 'model_fit_times': {'LightGBM_BAG_L1/T1': 7.457866430282593,
  'LightGBM_BAG_L1/T2': 6.54890513420105,
  'LightGBM_BAG_L1/T3': 6.900731563568115,
  'LightGBM_BAG_L1/T4': 6.897572994232178,
  'LightGBM_BAG_L1/T5': 7.732576370239258,
  'NeuralNetTorch_BAG_L1/T1': 25.508792400360107,
  'NeuralNetTorch_BAG_L1/T2': 38.108442068099976,
  'NeuralNetTorch_BAG_L1/T3': 35.47148656845093,
  'NeuralNetTorch_BAG_L1/T4': 72.15449714660645,
  'NeuralNetTorch_BAG_L1/T5': 88.62915325164795,
  'WeightedEnsemble_L2': 0.7970461845397949,
  'LightGBM_BAG_L2/T1': 7.921231031417847,
  'LightGBM_BAG_L2/T2': 7.105218887329102,
  'LightGBM_BAG_L2/T3': 7.991039037704468,
  'LightGBM_BAG_L2/T4': 7.180105447769165,
  'LightGBM_BAG_L2/T5': 7.912818193435669,
  'NeuralNetTorch_BAG_L2/T1': 25.693060636520386,
  'NeuralNetTorch_BAG_L2/T2': 36.16522741317749,
  'NeuralNetTorch_BAG_L2/T3': 35.405948638916016,
  'NeuralNetTorch_BAG_L2/T4': 60.85235857963562,
  'NeuralNetTorch_BAG_L2/T5': 81.01348757743835,
  'WeightedEnsemble_L3': 0.8742287158966064},
 'model_pred_times': {'LightGBM_BAG_L1/T1': 0.11072778701782227,
  'LightGBM_BAG_L1/T2': 0.12961769104003906,
  'LightGBM_BAG_L1/T3': 0.13564085960388184,
  'LightGBM_BAG_L1/T4': 0.0945291519165039,
  'LightGBM_BAG_L1/T5': 0.11731266975402832,
  'NeuralNetTorch_BAG_L1/T1': 0.3858356475830078,
  'NeuralNetTorch_BAG_L1/T2': 0.3144950866699219,
  'NeuralNetTorch_BAG_L1/T3': 0.33666515350341797,
  'NeuralNetTorch_BAG_L1/T4': 0.42746949195861816,
  'NeuralNetTorch_BAG_L1/T5': 0.48789024353027344,
  'WeightedEnsemble_L2': 0.0008318424224853516,
  'LightGBM_BAG_L2/T1': 0.10253620147705078,
  'LightGBM_BAG_L2/T2': 0.09523439407348633,
  'LightGBM_BAG_L2/T3': 0.14042353630065918,
  'LightGBM_BAG_L2/T4': 0.08096933364868164,
  'LightGBM_BAG_L2/T5': 0.11015439033508301,
  'NeuralNetTorch_BAG_L2/T1': 0.32818102836608887,
  'NeuralNetTorch_BAG_L2/T2': 0.3438386917114258,
  'NeuralNetTorch_BAG_L2/T3': 0.4233055114746094,
  'NeuralNetTorch_BAG_L2/T4': 0.6047353744506836,
  'NeuralNetTorch_BAG_L2/T5': 0.7249743938446045,
  'WeightedEnsemble_L3': 0.0013158321380615234},
 'num_bag_folds': 5,
 'max_stack_level': 3,
 'model_hyperparams': {'LightGBM_BAG_L1/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L2': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L3': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True}},
 'leaderboard':                        model   score_val  pred_time_val    fit_time  \
 0        WeightedEnsemble_L3  -40.706686       5.304729  558.431824   
 1         LightGBM_BAG_L2/T2  -41.037416       2.635418  302.515243   
 2         LightGBM_BAG_L2/T1  -41.200410       2.642720  303.331255   
 3         LightGBM_BAG_L2/T3  -41.346167       2.680607  303.401063   
 4         LightGBM_BAG_L2/T5  -41.647512       2.650338  303.322842   
 5        WeightedEnsemble_L2  -42.255533       1.008055  124.509622   
 6   NeuralNetTorch_BAG_L2/T1  -42.577094       2.868365  321.103085   
 7   NeuralNetTorch_BAG_L2/T4  -42.582856       3.144919  356.262383   
 8   NeuralNetTorch_BAG_L2/T2  -42.644782       2.884022  331.575251   
 9         LightGBM_BAG_L1/T2  -42.847297       0.129618    6.548905   
 10        LightGBM_BAG_L1/T3  -43.652435       0.135641    6.900732   
 11  NeuralNetTorch_BAG_L2/T5  -43.880374       3.265158  376.423512   
 12        LightGBM_BAG_L1/T1  -44.719007       0.110728    7.457866   
 13  NeuralNetTorch_BAG_L2/T3  -45.610407       2.963489  330.815973   
 14        LightGBM_BAG_L1/T5  -47.844107       0.117313    7.732576   
 15  NeuralNetTorch_BAG_L1/T4  -53.814540       0.427469   72.154497   
 16  NeuralNetTorch_BAG_L1/T2  -56.923024       0.314495   38.108442   
 17  NeuralNetTorch_BAG_L1/T3  -81.995966       0.336665   35.471487   
 18  NeuralNetTorch_BAG_L1/T1  -96.490001       0.385836   25.508792   
 19        LightGBM_BAG_L2/T4 -104.120960       2.621153  302.590129   
 20        LightGBM_BAG_L1/T4 -124.349788       0.094529    6.897573   
 21  NeuralNetTorch_BAG_L1/T5 -140.663465       0.487890   88.629153   
 
     pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  \
 0                 0.001316           0.874229            3       True   
 1                 0.095234           7.105219            2       True   
 2                 0.102536           7.921231            2       True   
 3                 0.140424           7.991039            2       True   
 4                 0.110154           7.912818            2       True   
 5                 0.000832           0.797046            2       True   
 6                 0.328181          25.693061            2       True   
 7                 0.604735          60.852359            2       True   
 8                 0.343839          36.165227            2       True   
 9                 0.129618           6.548905            1       True   
 10                0.135641           6.900732            1       True   
 11                0.724974          81.013488            2       True   
 12                0.110728           7.457866            1       True   
 13                0.423306          35.405949            2       True   
 14                0.117313           7.732576            1       True   
 15                0.427469          72.154497            1       True   
 16                0.314495          38.108442            1       True   
 17                0.336665          35.471487            1       True   
 18                0.385836          25.508792            1       True   
 19                0.080969           7.180105            2       True   
 20                0.094529           6.897573            1       True   
 21                0.487890          88.629153            1       True   
 
     fit_order  
 0          22  
 1          13  
 2          12  
 3          14  
 4          16  
 5          11  
 6          17  
 7          20  
 8          18  
 9           2  
 10          3  
 11         21  
 12          1  
 13         19  
 14          5  
 15          9  
 16          7  
 17          8  
 18          6  
 19         15  
 20          4  
 21         10  }

Create predictions for train dataset

In [54]:
#predictor_new_hpo
predictor_new_hpo_train = predictor_new_hpo.predict_proba(train)

rmse_new_hpo = mean_squared_error(train['count'], predictor_new_hpo_train, squared=False)
print('The RMSE for predictor new hpo is: ',rmse_new_hpo)
The RMSE for predictor new hpo is:  33.322557743463065

Create predictions for test dataset

In [55]:
#predictor_new_hpo
predictions_new_hpo = predictor_new_hpo.predict_proba(test)
In [56]:
submission_new_hpo = pd.DataFrame({'datetime': submission['datetime'],
                           'count': predictions_new_hpo})

submission_new_hpo.loc[submission_new_hpo['count'] < 0, 'count'] = 0
In [57]:
# Same submitting predictions
submission_new_hpo['count'] = submission_new_hpo['count'].fillna(0).astype(int)
submission_new_hpo['count'] = submission_new_hpo['count'].astype(np.int64)

submission_new_hpo.to_csv("submission_new_hpo.csv", index=False)
In [58]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_hpo.csv -m "new features with hyperparameters"
100%|█████████████████████████████████████████| 149k/149k [00:00<00:00, 199kB/s]
Successfully submitted to Bike Sharing Demand
In [59]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName                     date                 description                        status    publicScore  privateScore  
---------------------------  -------------------  ---------------------------------  --------  -----------  ------------  
submission_new_hpo.csv       2022-05-21 12:42:39  new features with hyperparameters  complete  0.47659      0.47659       
submission_new_features.csv  2022-05-21 12:28:51  new features                       complete  0.46603      0.46603       
submission.csv               2022-05-21 12:08:15  first raw submission               complete  1.82081      1.82081       
submission_new_hpo.csv       2022-05-19 10:02:34  new features with hyperparameters  complete  0.46282      0.46282       

New Score of 0.47659

Trying different combinations of Hyperparameters - version 2

In [79]:
predictor_new_hpo_v2 = TabularPredictor(
    label="count",
    eval_metric='root_mean_squared_error',
    learner_kwargs={'ignored_columns': ["casual", "registered"]}
).fit(
    train_data=train,
    time_limit=1200,
    num_bag_folds=6,
    num_bag_sets=1,
    num_stack_levels=2,
    presets="best_quality",
    hyperparameters = {'NN_TORCH': {'num_epochs': 2}, 'GBM': {'num_boost_round': 20}},
    hyperparameter_tune_kwargs='auto',
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_134143/"
Presets specified: ['best_quality']
Warning: hyperparameter tuning is currently experimental and may cause the process to hang.
Beginning AutoGluon training ... Time limit = 1200s
AutoGluon will save models to "AutogluonModels/ag-20220521_134143/"
AutoGluon Version:  0.4.1
Python Version:     3.7.10
Operating System:   Linux
Train Data Rows:    10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
AutoGluon infers your prediction problem is: 'regression' (because dtype of label-column == int and many unique label-values observed).
	Label info (max, min, mean, stddev): (977, 1, 191.57413, 181.14445)
	If 'regression' is not the correct problem_type, please manually specify the problem_type parameter during predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression'])
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
	Available Memory:                    1967.59 MB
	Train Data (Original)  Memory Usage: 0.74 MB (0.0% of available memory)
	Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
	Stage 1 Generators:
		Fitting AsTypeFeatureGenerator...
			Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
	Stage 2 Generators:
		Fitting FillNaFeatureGenerator...
	Stage 3 Generators:
		Fitting IdentityFeatureGenerator...
		Fitting CategoryFeatureGenerator...
			Fitting CategoryMemoryMinimizeFeatureGenerator...
	Stage 4 Generators:
		Fitting DropUniqueFeatureGenerator...
	Types of features in original data (raw dtype, special dtypes):
		('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
		('float', [])    : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])      : 5 | ['humidity', 'year', 'month', 'day', 'hour']
	Types of features in processed data (raw dtype, special dtypes):
		('category', [])  : 2 | ['season', 'weather']
		('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
		('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
		('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
	0.1s = Fit runtime
	12 features in original data used to generate 12 features in processed data.
	Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.16s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
	To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 3 stack levels (L1 to L3) ...
Fitting 2 L1 models ...
Hyperparameter tuning model: LightGBM_BAG_L1 ... Tuning model for up to 39.98s of the 1199.84s of remaining time.
	Ran out of time, early stopping on iteration 1. Best iteration is:
	[1]	valid_set's rmse: 180.547
	Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L1/T1 ...
	-96.1594	 = Validation score   (root_mean_squared_error)
	0.25s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T2 ...
	-71.2354	 = Validation score   (root_mean_squared_error)
	0.29s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T3 ...
	-90.8859	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T4 ...
	-168.1893	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T5 ...
	-62.8971	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T6 ...
	-137.0979	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T7 ...
	-61.584	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T8 ...
	-61.6155	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T9 ...
	-162.6852	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T10 ...
	-47.2738	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T11 ...
	-152.0219	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T12 ...
	-163.3942	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T13 ...
	-156.4626	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T14 ...
	-134.9397	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T15 ...
	-168.3672	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T16 ...
	-163.8489	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T17 ...
	-168.9279	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T18 ...
	-98.1435	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T19 ...
	-103.6901	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T20 ...
	-102.5023	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T21 ...
	-143.9416	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T22 ...
	-100.2582	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T23 ...
	-97.9426	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T24 ...
	-48.9969	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T25 ...
	-168.3117	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T26 ...
	-149.2756	 = Validation score   (root_mean_squared_error)
	0.25s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T27 ...
	-167.9788	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T28 ...
	-152.403	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T29 ...
	-106.8089	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T30 ...
	-59.483	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T31 ...
	-148.4568	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T32 ...
	-160.2138	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T33 ...
	-113.0535	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T34 ...
	-67.9513	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T35 ...
	-83.579	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T36 ...
	-63.8211	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T37 ...
	-60.5162	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T38 ...
	-101.2134	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T39 ...
	-84.9816	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T40 ...
	-135.251	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T41 ...
	-92.298	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T42 ...
	-127.3798	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T43 ...
	-119.9997	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T44 ...
	-83.7239	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T45 ...
	-163.3203	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T46 ...
	-57.4981	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T47 ...
	-48.4945	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T48 ...
	-163.7653	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T49 ...
	-159.4538	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T50 ...
	-128.7126	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T51 ...
	-56.6156	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T52 ...
	-168.9781	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T53 ...
	-156.6159	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T54 ...
	-67.2121	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T55 ...
	-150.6513	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T56 ...
	-154.0453	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T57 ...
	-98.83	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T58 ...
	-100.567	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T59 ...
	-93.4209	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T60 ...
	-54.9384	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T61 ...
	-47.8343	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T62 ...
	-75.0451	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T63 ...
	-125.976	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T64 ...
	-89.8658	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T65 ...
	-84.805	 = Validation score   (root_mean_squared_error)
	0.49s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T66 ...
	-93.101	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T67 ...
	-53.1081	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T68 ...
	-142.6263	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T69 ...
	-57.8068	 = Validation score   (root_mean_squared_error)
	0.27s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T70 ...
	-146.4761	 = Validation score   (root_mean_squared_error)
	0.25s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T71 ...
	-45.0568	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T72 ...
	-52.3685	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T73 ...
	-158.3782	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T74 ...
	-158.1351	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T75 ...
	-123.3684	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T76 ...
	-53.1857	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T77 ...
	-162.2208	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T78 ...
	-56.8812	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T79 ...
	-165.8162	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T80 ...
	-164.8504	 = Validation score   (root_mean_squared_error)
	0.27s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T81 ...
	-73.609	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T82 ...
	-156.9424	 = Validation score   (root_mean_squared_error)
	0.3s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T83 ...
	-55.5956	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T84 ...
	-146.8305	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T85 ...
	-97.4384	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T86 ...
	-154.4029	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T87 ...
	-113.4807	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T88 ...
	-114.6658	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T89 ...
	-46.2614	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T90 ...
	-49.4682	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T91 ...
	-136.4768	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T92 ...
	-116.418	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T93 ...
	-56.8044	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T94 ...
	-48.5038	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T95 ...
	-166.6625	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T96 ...
	-164.8862	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T97 ...
	-171.4948	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T98 ...
	-169.8671	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T99 ...
	-151.3003	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T100 ...
	-50.9384	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T101 ...
	-163.6301	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T102 ...
	-124.9231	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T103 ...
	-137.3387	 = Validation score   (root_mean_squared_error)
	0.24s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T104 ...
	-100.4694	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T105 ...
	-154.2182	 = Validation score   (root_mean_squared_error)
	0.21s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T106 ...
	-53.0285	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T107 ...
	-45.9895	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T108 ...
	-126.3663	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T109 ...
	-46.4378	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T110 ...
	-138.44	 = Validation score   (root_mean_squared_error)
	0.2s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T111 ...
	-78.7875	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T112 ...
	-138.63	 = Validation score   (root_mean_squared_error)
	0.22s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T113 ...
	-50.3731	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T114 ...
	-51.1664	 = Validation score   (root_mean_squared_error)
	0.23s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L1/T115 ...
	-180.5469	 = Validation score   (root_mean_squared_error)
	0.19s	 = Training   runtime
	0.01s	 = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L1 ... Tuning model for up to 39.98s of the 1155.85s of remaining time.
	Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L1/T1 ...
	-138.793	 = Validation score   (root_mean_squared_error)
	0.94s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T2 ...
	-105.4212	 = Validation score   (root_mean_squared_error)
	1.64s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T3 ...
	-121.3764	 = Validation score   (root_mean_squared_error)
	1.37s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T4 ...
	-102.7891	 = Validation score   (root_mean_squared_error)
	1.22s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T5 ...
	-124.832	 = Validation score   (root_mean_squared_error)
	2.31s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T6 ...
	-96.9827	 = Validation score   (root_mean_squared_error)
	1.5s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T7 ...
	-148.9562	 = Validation score   (root_mean_squared_error)
	1.12s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T8 ...
	-132.9172	 = Validation score   (root_mean_squared_error)
	0.99s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T9 ...
	-127.5594	 = Validation score   (root_mean_squared_error)
	1.2s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T10 ...
	-127.8206	 = Validation score   (root_mean_squared_error)
	1.22s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T11 ...
	-99.7462	 = Validation score   (root_mean_squared_error)
	1.88s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T12 ...
	-105.7916	 = Validation score   (root_mean_squared_error)
	0.86s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T13 ...
	-135.9016	 = Validation score   (root_mean_squared_error)
	3.42s	 = Training   runtime
	0.06s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T14 ...
	-140.0589	 = Validation score   (root_mean_squared_error)
	1.0s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T15 ...
	-97.6155	 = Validation score   (root_mean_squared_error)
	2.22s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T16 ...
	-109.0504	 = Validation score   (root_mean_squared_error)
	2.86s	 = Training   runtime
	0.05s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T17 ...
	-129.0148	 = Validation score   (root_mean_squared_error)
	1.15s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T18 ...
	-143.8835	 = Validation score   (root_mean_squared_error)
	2.76s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 452.46s of the 1119.17s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-95.2464	 = Validation score   (root_mean_squared_error)
	10.0s	 = Training   runtime
	0.04s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 439.53s of the 1106.24s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-70.4303	 = Validation score   (root_mean_squared_error)
	9.12s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 427.96s of the 1094.67s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-89.9804	 = Validation score   (root_mean_squared_error)
	9.34s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T4 ... Training model for up to 415.56s of the 1082.27s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-165.429	 = Validation score   (root_mean_squared_error)
	9.58s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T5 ... Training model for up to 402.97s of the 1069.68s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-62.1499	 = Validation score   (root_mean_squared_error)
	8.97s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T6 ... Training model for up to 391.61s of the 1058.32s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-134.9459	 = Validation score   (root_mean_squared_error)
	9.39s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T7 ... Training model for up to 379.79s of the 1046.5s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-61.4308	 = Validation score   (root_mean_squared_error)
	9.29s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T8 ... Training model for up to 367.38s of the 1034.09s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-61.5958	 = Validation score   (root_mean_squared_error)
	9.49s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T9 ... Training model for up to 355.13s of the 1021.84s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-160.2092	 = Validation score   (root_mean_squared_error)
	9.1s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T10 ... Training model for up to 343.71s of the 1010.42s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-46.9856	 = Validation score   (root_mean_squared_error)
	9.39s	 = Training   runtime
	0.09s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T11 ... Training model for up to 331.98s of the 998.69s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-149.4088	 = Validation score   (root_mean_squared_error)
	10.0s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T12 ... Training model for up to 318.82s of the 985.53s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-160.8696	 = Validation score   (root_mean_squared_error)
	9.38s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T13 ... Training model for up to 306.47s of the 973.18s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-154.0386	 = Validation score   (root_mean_squared_error)
	9.2s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T14 ... Training model for up to 294.88s of the 961.59s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-132.4655	 = Validation score   (root_mean_squared_error)
	9.06s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T15 ... Training model for up to 283.42s of the 950.13s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-165.8398	 = Validation score   (root_mean_squared_error)
	10.26s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T16 ... Training model for up to 269.98s of the 936.69s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-161.0969	 = Validation score   (root_mean_squared_error)
	10.56s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T17 ... Training model for up to 256.37s of the 923.08s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-166.2581	 = Validation score   (root_mean_squared_error)
	9.65s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T18 ... Training model for up to 244.08s of the 910.79s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-98.2333	 = Validation score   (root_mean_squared_error)
	9.48s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T19 ... Training model for up to 232.04s of the 898.75s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-102.7429	 = Validation score   (root_mean_squared_error)
	9.56s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T20 ... Training model for up to 220.01s of the 886.72s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-101.2778	 = Validation score   (root_mean_squared_error)
	10.57s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T21 ... Training model for up to 206.41s of the 873.12s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-141.5044	 = Validation score   (root_mean_squared_error)
	10.07s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T22 ... Training model for up to 193.81s of the 860.52s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-99.9158	 = Validation score   (root_mean_squared_error)
	9.56s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T23 ... Training model for up to 181.73s of the 848.44s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-96.7825	 = Validation score   (root_mean_squared_error)
	10.14s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T24 ... Training model for up to 168.4s of the 835.11s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-48.8118	 = Validation score   (root_mean_squared_error)
	9.29s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T25 ... Training model for up to 156.61s of the 823.32s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-165.7382	 = Validation score   (root_mean_squared_error)
	9.39s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T26 ... Training model for up to 144.03s of the 810.74s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-146.8745	 = Validation score   (root_mean_squared_error)
	10.2s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T27 ... Training model for up to 130.5s of the 797.21s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-165.2447	 = Validation score   (root_mean_squared_error)
	10.12s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T28 ... Training model for up to 117.38s of the 784.09s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-150.1905	 = Validation score   (root_mean_squared_error)
	9.47s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T29 ... Training model for up to 105.54s of the 772.25s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-105.026	 = Validation score   (root_mean_squared_error)
	10.45s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T30 ... Training model for up to 92.07s of the 758.78s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-59.3014	 = Validation score   (root_mean_squared_error)
	9.83s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T31 ... Training model for up to 78.69s of the 745.4s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-146.3214	 = Validation score   (root_mean_squared_error)
	10.55s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T32 ... Training model for up to 65.0s of the 731.71s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-157.5559	 = Validation score   (root_mean_squared_error)
	9.77s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T33 ... Training model for up to 51.93s of the 718.64s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-111.5454	 = Validation score   (root_mean_squared_error)
	9.89s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T34 ... Training model for up to 38.95s of the 705.66s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-67.2108	 = Validation score   (root_mean_squared_error)
	10.79s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T35 ... Training model for up to 25.0s of the 691.71s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-82.7133	 = Validation score   (root_mean_squared_error)
	9.59s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T36 ... Training model for up to 12.89s of the 679.6s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-63.9508	 = Validation score   (root_mean_squared_error)
	9.49s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L1/T37 ... Training model for up to 0.34s of the 667.05s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-152.9645	 = Validation score   (root_mean_squared_error)
	9.95s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 651.19s of remaining time.
	-46.7793	 = Validation score   (root_mean_squared_error)
	0.48s	 = Training   runtime
	0.0s	 = Validation runtime
Fitting 2 L2 models ...
Hyperparameter tuning model: LightGBM_BAG_L2 ... Tuning model for up to 32.52s of the 650.55s of remaining time.
	Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L2/T1 ...
	-77.8056	 = Validation score   (root_mean_squared_error)
	0.42s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T2 ...
	-61.3433	 = Validation score   (root_mean_squared_error)
	0.51s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T3 ...
	-77.7141	 = Validation score   (root_mean_squared_error)
	0.44s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T4 ...
	-161.3252	 = Validation score   (root_mean_squared_error)
	0.47s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T5 ...
	-50.6378	 = Validation score   (root_mean_squared_error)
	0.43s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T6 ...
	-131.3877	 = Validation score   (root_mean_squared_error)
	0.54s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T7 ...
	-53.5851	 = Validation score   (root_mean_squared_error)
	0.49s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T8 ...
	-51.9891	 = Validation score   (root_mean_squared_error)
	0.42s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T9 ...
	-156.7206	 = Validation score   (root_mean_squared_error)
	0.5s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T10 ...
	-46.4821	 = Validation score   (root_mean_squared_error)
	0.47s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T11 ...
	-141.5367	 = Validation score   (root_mean_squared_error)
	0.47s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T12 ...
	-155.3037	 = Validation score   (root_mean_squared_error)
	0.44s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T13 ...
	-146.6177	 = Validation score   (root_mean_squared_error)
	0.54s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T14 ...
	-112.6792	 = Validation score   (root_mean_squared_error)
	0.36s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T15 ...
	-161.1489	 = Validation score   (root_mean_squared_error)
	0.45s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T16 ...
	-155.7941	 = Validation score   (root_mean_squared_error)
	0.79s	 = Training   runtime
	0.02s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T17 ...
	-162.5445	 = Validation score   (root_mean_squared_error)
	0.64s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T18 ...
	-72.6803	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T19 ...
	-86.2001	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T20 ...
	-77.9314	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T21 ...
	-134.4647	 = Validation score   (root_mean_squared_error)
	0.46s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T22 ...
	-71.052	 = Validation score   (root_mean_squared_error)
	0.36s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T23 ...
	-68.9161	 = Validation score   (root_mean_squared_error)
	0.36s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T24 ...
	-46.1152	 = Validation score   (root_mean_squared_error)
	0.38s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T25 ...
	-162.5596	 = Validation score   (root_mean_squared_error)
	0.48s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T26 ...
	-137.4156	 = Validation score   (root_mean_squared_error)
	0.47s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T27 ...
	-160.8301	 = Validation score   (root_mean_squared_error)
	0.45s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T28 ...
	-141.4941	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T29 ...
	-87.1743	 = Validation score   (root_mean_squared_error)
	0.46s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T30 ...
	-48.811	 = Validation score   (root_mean_squared_error)
	0.4s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T31 ...
	-141.5471	 = Validation score   (root_mean_squared_error)
	0.45s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T32 ...
	-152.5716	 = Validation score   (root_mean_squared_error)
	0.42s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T33 ...
	-88.7714	 = Validation score   (root_mean_squared_error)
	0.38s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T34 ...
	-55.7494	 = Validation score   (root_mean_squared_error)
	0.48s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T35 ...
	-74.4511	 = Validation score   (root_mean_squared_error)
	0.5s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T36 ...
	-49.0472	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T37 ...
	-50.4294	 = Validation score   (root_mean_squared_error)
	0.5s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T38 ...
	-86.2722	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T39 ...
	-70.7915	 = Validation score   (root_mean_squared_error)
	0.48s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T40 ...
	-122.1557	 = Validation score   (root_mean_squared_error)
	0.47s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T41 ...
	-70.929	 = Validation score   (root_mean_squared_error)
	0.4s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T42 ...
	-116.2684	 = Validation score   (root_mean_squared_error)
	0.49s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T43 ...
	-88.9744	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T44 ...
	-71.7539	 = Validation score   (root_mean_squared_error)
	0.49s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T45 ...
	-154.7645	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T46 ...
	-50.7888	 = Validation score   (root_mean_squared_error)
	0.46s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T47 ...
	-46.8989	 = Validation score   (root_mean_squared_error)
	0.44s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T48 ...
	-154.1139	 = Validation score   (root_mean_squared_error)
	0.36s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T49 ...
	-152.4578	 = Validation score   (root_mean_squared_error)
	0.44s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T50 ...
	-119.415	 = Validation score   (root_mean_squared_error)
	0.44s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T51 ...
	-46.6812	 = Validation score   (root_mean_squared_error)
	0.34s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L2/T52 ...
	-161.7798	 = Validation score   (root_mean_squared_error)
	0.42s	 = Training   runtime
	0.01s	 = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L2 ... Tuning model for up to 32.52s of the 619.09s of remaining time.
	Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L2/T1 ...
	-52.6743	 = Validation score   (root_mean_squared_error)
	1.28s	 = Training   runtime
	0.08s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T2 ...
	-50.9213	 = Validation score   (root_mean_squared_error)
	1.73s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T3 ...
	-67.2741	 = Validation score   (root_mean_squared_error)
	1.93s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T4 ...
	-53.2503	 = Validation score   (root_mean_squared_error)
	1.48s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T5 ...
	-62.0811	 = Validation score   (root_mean_squared_error)
	2.44s	 = Training   runtime
	0.28s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T6 ...
	-52.6096	 = Validation score   (root_mean_squared_error)
	1.77s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T7 ...
	-52.95	 = Validation score   (root_mean_squared_error)
	1.22s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T8 ...
	-51.4724	 = Validation score   (root_mean_squared_error)
	1.29s	 = Training   runtime
	0.08s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T9 ...
	-51.1214	 = Validation score   (root_mean_squared_error)
	1.47s	 = Training   runtime
	0.08s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T10 ...
	-60.8847	 = Validation score   (root_mean_squared_error)
	1.64s	 = Training   runtime
	0.08s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T11 ...
	-47.6594	 = Validation score   (root_mean_squared_error)
	2.26s	 = Training   runtime
	0.09s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T12 ...
	-51.9495	 = Validation score   (root_mean_squared_error)
	0.93s	 = Training   runtime
	0.03s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T13 ...
	-98.2698	 = Validation score   (root_mean_squared_error)
	3.53s	 = Training   runtime
	0.1s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T1 ... Training model for up to 373.43s of the 590.35s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-77.1487	 = Validation score   (root_mean_squared_error)
	11.52s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T2 ... Training model for up to 359.05s of the 575.96s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-60.1188	 = Validation score   (root_mean_squared_error)
	10.67s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T3 ... Training model for up to 346.07s of the 562.98s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-77.0813	 = Validation score   (root_mean_squared_error)
	10.85s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T4 ... Training model for up to 332.81s of the 549.72s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-161.4144	 = Validation score   (root_mean_squared_error)
	10.94s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T5 ... Training model for up to 319.09s of the 536.01s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-48.5297	 = Validation score   (root_mean_squared_error)
	10.92s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T6 ... Training model for up to 305.24s of the 522.16s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-131.3732	 = Validation score   (root_mean_squared_error)
	10.88s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T7 ... Training model for up to 291.69s of the 508.6s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-51.7099	 = Validation score   (root_mean_squared_error)
	10.53s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T8 ... Training model for up to 278.58s of the 495.49s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-49.7948	 = Validation score   (root_mean_squared_error)
	10.6s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T9 ... Training model for up to 265.81s of the 482.72s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-156.791	 = Validation score   (root_mean_squared_error)
	10.96s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T10 ... Training model for up to 251.93s of the 468.83s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-43.937	 = Validation score   (root_mean_squared_error)
	11.39s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T11 ... Training model for up to 237.45s of the 454.36s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-141.5751	 = Validation score   (root_mean_squared_error)
	10.84s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T12 ... Training model for up to 223.64s of the 440.55s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-155.3657	 = Validation score   (root_mean_squared_error)
	10.33s	 = Training   runtime
	0.09s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T13 ... Training model for up to 211.12s of the 428.03s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-146.6166	 = Validation score   (root_mean_squared_error)
	10.76s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T14 ... Training model for up to 197.53s of the 414.44s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-112.6371	 = Validation score   (root_mean_squared_error)
	10.41s	 = Training   runtime
	0.11s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T15 ... Training model for up to 184.86s of the 401.76s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-161.265	 = Validation score   (root_mean_squared_error)
	10.21s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T16 ... Training model for up to 171.48s of the 388.4s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-155.8769	 = Validation score   (root_mean_squared_error)
	11.17s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T17 ... Training model for up to 157.71s of the 374.62s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-162.6224	 = Validation score   (root_mean_squared_error)
	11.74s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T18 ... Training model for up to 143.13s of the 360.04s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-71.8659	 = Validation score   (root_mean_squared_error)
	10.41s	 = Training   runtime
	0.05s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T19 ... Training model for up to 129.55s of the 346.46s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-85.8107	 = Validation score   (root_mean_squared_error)
	11.45s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T20 ... Training model for up to 114.77s of the 331.68s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-77.2805	 = Validation score   (root_mean_squared_error)
	10.2s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T21 ... Training model for up to 102.01s of the 318.92s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-134.4673	 = Validation score   (root_mean_squared_error)
	10.5s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T22 ... Training model for up to 89.01s of the 305.92s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-70.1588	 = Validation score   (root_mean_squared_error)
	10.57s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T23 ... Training model for up to 75.86s of the 292.77s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-67.9272	 = Validation score   (root_mean_squared_error)
	10.37s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T24 ... Training model for up to 63.05s of the 279.96s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-43.571	 = Validation score   (root_mean_squared_error)
	10.31s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T25 ... Training model for up to 50.3s of the 267.21s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-162.6221	 = Validation score   (root_mean_squared_error)
	11.15s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T26 ... Training model for up to 36.19s of the 253.08s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-137.4171	 = Validation score   (root_mean_squared_error)
	11.17s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T27 ... Training model for up to 21.91s of the 238.82s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-160.9142	 = Validation score   (root_mean_squared_error)
	10.47s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: LightGBM_BAG_L2/T28 ... Training model for up to 8.97s of the 225.88s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-141.6145	 = Validation score   (root_mean_squared_error)
	11.03s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the 209.89s of remaining time.
	-43.4603	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.0s	 = Validation runtime
Fitting 2 L3 models ...
Hyperparameter tuning model: LightGBM_BAG_L3 ... Tuning model for up to 15.71s of the 209.39s of remaining time.
	Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L3/T1 ...
	-76.7346	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T2 ...
	-61.0555	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T3 ...
	-76.9387	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T4 ...
	-160.8138	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T5 ...
	-50.0197	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T6 ...
	-130.7797	 = Validation score   (root_mean_squared_error)
	0.43s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T7 ...
	-52.9297	 = Validation score   (root_mean_squared_error)
	0.45s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T8 ...
	-50.989	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T9 ...
	-156.1403	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T10 ...
	-46.3789	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T11 ...
	-140.9668	 = Validation score   (root_mean_squared_error)
	0.38s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T12 ...
	-154.6995	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T13 ...
	-145.9833	 = Validation score   (root_mean_squared_error)
	0.41s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T14 ...
	-111.7448	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T15 ...
	-160.5393	 = Validation score   (root_mean_squared_error)
	0.33s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T16 ...
	-155.2734	 = Validation score   (root_mean_squared_error)
	0.38s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T17 ...
	-162.0223	 = Validation score   (root_mean_squared_error)
	0.4s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T18 ...
	-71.4082	 = Validation score   (root_mean_squared_error)
	0.33s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T19 ...
	-85.3119	 = Validation score   (root_mean_squared_error)
	0.33s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T20 ...
	-76.8438	 = Validation score   (root_mean_squared_error)
	0.33s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T21 ...
	-133.8836	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T22 ...
	-69.6403	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T23 ...
	-67.5269	 = Validation score   (root_mean_squared_error)
	0.31s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T24 ...
	-45.7577	 = Validation score   (root_mean_squared_error)
	0.33s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T25 ...
	-161.9758	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T26 ...
	-136.779	 = Validation score   (root_mean_squared_error)
	0.39s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T27 ...
	-160.2776	 = Validation score   (root_mean_squared_error)
	0.37s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T28 ...
	-140.8378	 = Validation score   (root_mean_squared_error)
	0.34s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T29 ...
	-86.5646	 = Validation score   (root_mean_squared_error)
	0.36s	 = Training   runtime
	0.01s	 = Validation runtime
Fitted model: LightGBM_BAG_L3/T30 ...
	-48.1206	 = Validation score   (root_mean_squared_error)
	0.34s	 = Training   runtime
	0.01s	 = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L3 ... Tuning model for up to 15.71s of the 193.81s of remaining time.
	Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L3/T1 ...
	-48.5864	 = Validation score   (root_mean_squared_error)
	1.23s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T2 ...
	-50.2672	 = Validation score   (root_mean_squared_error)
	1.7s	 = Training   runtime
	0.04s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T3 ...
	-59.9232	 = Validation score   (root_mean_squared_error)
	1.58s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T4 ...
	-51.119	 = Validation score   (root_mean_squared_error)
	1.46s	 = Training   runtime
	0.07s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T5 ...
	-52.6298	 = Validation score   (root_mean_squared_error)
	2.43s	 = Training   runtime
	0.08s	 = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T6 ...
	-54.5518	 = Validation score   (root_mean_squared_error)
	1.92s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T1 ... Training model for up to 180.96s of the 180.91s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-76.6953	 = Validation score   (root_mean_squared_error)
	10.42s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T2 ... Training model for up to 167.16s of the 167.11s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-60.3636	 = Validation score   (root_mean_squared_error)
	10.75s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T3 ... Training model for up to 153.84s of the 153.79s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-76.8015	 = Validation score   (root_mean_squared_error)
	10.53s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T4 ... Training model for up to 140.23s of the 140.18s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-161.3706	 = Validation score   (root_mean_squared_error)
	10.47s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T5 ... Training model for up to 126.44s of the 126.39s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-48.6524	 = Validation score   (root_mean_squared_error)
	10.82s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T6 ... Training model for up to 112.34s of the 112.29s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-131.2738	 = Validation score   (root_mean_squared_error)
	14.98s	 = Training   runtime
	0.08s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T7 ... Training model for up to 94.28s of the 94.23s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-51.7651	 = Validation score   (root_mean_squared_error)
	10.51s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T8 ... Training model for up to 81.4s of the 81.35s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-49.6566	 = Validation score   (root_mean_squared_error)
	10.47s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T9 ... Training model for up to 67.92s of the 67.87s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-156.7114	 = Validation score   (root_mean_squared_error)
	11.09s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T10 ... Training model for up to 53.96s of the 53.91s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-44.3452	 = Validation score   (root_mean_squared_error)
	11.27s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T11 ... Training model for up to 39.33s of the 39.28s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-141.4654	 = Validation score   (root_mean_squared_error)
	10.66s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T12 ... Training model for up to 25.4s of the 25.35s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-155.2612	 = Validation score   (root_mean_squared_error)
	10.28s	 = Training   runtime
	0.06s	 = Validation runtime
Fitting model: LightGBM_BAG_L3/T13 ... Training model for up to 12.61s of the 12.56s of remaining time.
	Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
	-146.5451	 = Validation score   (root_mean_squared_error)
	10.35s	 = Training   runtime
	0.07s	 = Validation runtime
Fitting model: WeightedEnsemble_L4 ... Training model for up to 360.0s of the -1.63s of remaining time.
	-44.3452	 = Validation score   (root_mean_squared_error)
	0.35s	 = Training   runtime
	0.0s	 = Validation runtime
AutoGluon training complete, total runtime = 1202.27s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_134143/")
In [80]:
predictor_new_hpo_v2.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
                         model   score_val  pred_time_val    fit_time  pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  fit_order
0          WeightedEnsemble_L3  -43.460319       2.306577  381.974152                0.000876           0.347950            3       True        200
1          LightGBM_BAG_L2/T24  -43.570961       2.235956  370.239415                0.072493          10.308636            2       True        158
2          LightGBM_BAG_L2/T10  -43.937039       2.233209  371.317567                0.069745          11.386788            2       True        144
3          LightGBM_BAG_L3/T10  -44.345180       4.141941  673.566589                0.064293          11.269752            3       True        210
4          WeightedEnsemble_L4  -44.345180       4.142688  673.915158                0.000747           0.348568            4       True        237
5          LightGBM_BAG_L1/T71  -45.056774       0.008519    0.235917                0.008519           0.235917            1       True         71
6          LightGBM_BAG_L3/T24  -45.757680       4.084984  662.625002                0.007337           0.328164            3       True        224
7         LightGBM_BAG_L1/T107  -45.989459       0.007218    0.229095                0.007218           0.229095            1       True        107
8          LightGBM_BAG_L1/T89  -46.261418       0.007159    0.224034                0.007159           0.224034            1       True         89
9         LightGBM_BAG_L1/T109  -46.437759       0.007054    0.221164                0.007054           0.221164            1       True        109
10         LightGBM_BAG_L2/T51  -46.681152       2.170503  360.266608                0.007039           0.335829            2       True        185
11         WeightedEnsemble_L2  -46.779286       0.163266   19.163258                0.001148           0.475723            2       True        134
12         LightGBM_BAG_L2/T47  -46.898943       2.171569  360.366563                0.008106           0.435784            2       True        181
13         LightGBM_BAG_L1/T10  -46.985623       0.089329    9.394629                0.089329           9.394629            1       True         10
14   NeuralNetTorch_BAG_L2/T11  -47.659355       2.255722  362.188172                0.092258           2.257393            2       True        197
15         LightGBM_BAG_L1/T61  -47.834260       0.007396    0.232562                0.007396           0.232562            1       True         61
16         LightGBM_BAG_L3/T30  -48.120608       4.085141  662.634193                0.007494           0.337355            3       True        230
17         LightGBM_BAG_L1/T47  -48.494512       0.007655    0.225170                0.007655           0.225170            1       True         47
18         LightGBM_BAG_L1/T94  -48.503779       0.006870    0.220363                0.006870           0.220363            1       True         94
19          LightGBM_BAG_L2/T5  -48.529699       2.227863  370.850576                0.064399          10.919797            2       True        139
20    NeuralNetTorch_BAG_L3/T1  -48.586389       4.147132  663.528044                0.069485           1.231207            3       True        231
21          LightGBM_BAG_L3/T5  -48.652445       4.146297  673.116226                0.068650          10.819388            3       True        205
22         LightGBM_BAG_L2/T30  -48.811046       2.171432  360.326760                0.007968           0.395981            2       True        164
23         LightGBM_BAG_L1/T24  -48.811782       0.072790    9.292907                0.072790           9.292907            1       True         24
24         LightGBM_BAG_L2/T36  -49.047246       2.174662  360.296565                0.011198           0.365786            2       True        170
25         LightGBM_BAG_L1/T90  -49.468162       0.006792    0.206065                0.006792           0.206065            1       True         90
26          LightGBM_BAG_L3/T8  -49.656635       4.133121  672.771686                0.055474          10.474848            3       True        208
27          LightGBM_BAG_L2/T8  -49.794838       2.231861  370.528136                0.068398          10.597358            2       True        142
28    NeuralNetTorch_BAG_L3/T2  -50.267170       4.115871  663.997388                0.038224           1.700550            3       True        232
29        LightGBM_BAG_L1/T113  -50.373094       0.007121    0.230512                0.007121           0.230512            1       True        113
30         LightGBM_BAG_L2/T37  -50.429360       2.172501  360.431727                0.009037           0.500948            2       True        171
31         LightGBM_BAG_L2/T46  -50.788790       2.172910  360.387473                0.009446           0.456694            2       True        180
32    NeuralNetTorch_BAG_L2/T2  -50.921303       2.198969  361.662731                0.035506           1.731952            2       True        188
33        LightGBM_BAG_L1/T100  -50.938412       0.007288    0.228707                0.007288           0.228707            1       True        100
34    NeuralNetTorch_BAG_L3/T4  -51.118959       4.143447  663.759203                0.065799           1.462365            3       True        234
35    NeuralNetTorch_BAG_L2/T9  -51.121433       2.244301  361.399956                0.080838           1.469177            2       True        195
36        LightGBM_BAG_L1/T114  -51.166393       0.012602    0.228076                0.012602           0.228076            1       True        114
37    NeuralNetTorch_BAG_L2/T8  -51.472392       2.238962  361.224025                0.075498           1.293246            2       True        194
38          LightGBM_BAG_L2/T7  -51.709883       2.233213  370.459575                0.069749          10.528796            2       True        141
39          LightGBM_BAG_L3/T7  -51.765128       4.149523  672.806423                0.071875          10.509585            3       True        207
40   NeuralNetTorch_BAG_L2/T12  -51.949523       2.190379  360.860381                0.026915           0.929602            2       True        198
41         LightGBM_BAG_L1/T72  -52.368527       0.009573    0.212278                0.009573           0.212278            1       True         72
42    NeuralNetTorch_BAG_L2/T6  -52.609616       2.232281  361.699302                0.068818           1.768523            2       True        192
43    NeuralNetTorch_BAG_L3/T5  -52.629777       4.153352  664.726630                0.075705           2.429793            3       True        235
44    NeuralNetTorch_BAG_L2/T1  -52.674276       2.239338  361.214180                0.075874           1.283401            2       True        187
45    NeuralNetTorch_BAG_L2/T7  -52.950038       2.238297  361.153965                0.074833           1.223186            2       True        193
46        LightGBM_BAG_L1/T106  -53.028500       0.008013    0.222290                0.008013           0.222290            1       True        106
47         LightGBM_BAG_L1/T67  -53.108142       0.008053    0.244969                0.008053           0.244969            1       True         67
48         LightGBM_BAG_L1/T76  -53.185713       0.007575    0.220364                0.007575           0.220364            1       True         76
49    NeuralNetTorch_BAG_L2/T4  -53.250292       2.230874  361.407783                0.067410           1.477004            2       True        190
50    NeuralNetTorch_BAG_L3/T6  -54.551765       4.149120  664.217582                0.071473           1.920745            3       True        236
51         LightGBM_BAG_L1/T60  -54.938450       0.007218    0.210299                0.007218           0.210299            1       True         60
52         LightGBM_BAG_L1/T83  -55.595622       0.006056    0.207053                0.006056           0.207053            1       True         83
53         LightGBM_BAG_L2/T34  -55.749416       2.171851  360.406150                0.008388           0.475371            2       True        168
54         LightGBM_BAG_L1/T51  -56.615568       0.006451    0.204403                0.006451           0.204403            1       True         51
55         LightGBM_BAG_L1/T93  -56.804420       0.006592    0.222801                0.006592           0.222801            1       True         93
56         LightGBM_BAG_L1/T78  -56.881182       0.007219    0.226611                0.007219           0.226611            1       True         78
57         LightGBM_BAG_L1/T46  -57.498094       0.007246    0.231942                0.007246           0.231942            1       True         46
58         LightGBM_BAG_L1/T69  -57.806844       0.007620    0.269982                0.007620           0.269982            1       True         69
59         LightGBM_BAG_L1/T30  -59.301369       0.065185    9.833703                0.065185           9.833703            1       True         30
60    NeuralNetTorch_BAG_L3/T3  -59.923184       4.149328  663.878519                0.071681           1.581681            3       True        233
61          LightGBM_BAG_L2/T2  -60.118779       2.241281  370.604797                0.077817          10.674018            2       True        136
62          LightGBM_BAG_L3/T2  -60.363647       4.139754  673.042545                0.062107          10.745707            3       True        202
63   NeuralNetTorch_BAG_L2/T10  -60.884651       2.239229  361.569122                0.075765           1.638343            2       True        196
64          LightGBM_BAG_L1/T7  -61.430823       0.058785    9.292935                0.058785           9.292935            1       True          7
65          LightGBM_BAG_L1/T8  -61.595818       0.059454    9.493112                0.059454           9.493112            1       True          8
66    NeuralNetTorch_BAG_L2/T5  -62.081090       2.439783  362.374147                0.276320           2.443368            2       True        191
67          LightGBM_BAG_L1/T5  -62.149900       0.058731    8.969120                0.058731           8.969120            1       True          5
68         LightGBM_BAG_L1/T36  -63.950820       0.065624    9.486712                0.065624           9.486712            1       True         36
69         LightGBM_BAG_L1/T34  -67.210837       0.063193   10.786390                0.063193          10.786390            1       True         34
70         LightGBM_BAG_L1/T54  -67.212089       0.006682    0.218171                0.006682           0.218171            1       True         54
71    NeuralNetTorch_BAG_L2/T3  -67.274105       2.237578  361.859015                0.074114           1.928236            2       True        189
72         LightGBM_BAG_L3/T23  -67.526948       4.084053  662.609949                0.006406           0.313112            3       True        223
73         LightGBM_BAG_L2/T23  -67.927218       2.222276  370.304873                0.058812          10.374094            2       True        157
74         LightGBM_BAG_L3/T22  -69.640327       4.084576  662.607048                0.006929           0.310210            3       True        222
75         LightGBM_BAG_L2/T22  -70.158818       2.221691  370.503316                0.058227          10.572537            2       True        156
76          LightGBM_BAG_L1/T2  -70.430295       0.062552    9.124408                0.062552           9.124408            1       True          2
77         LightGBM_BAG_L2/T39  -70.791530       2.172120  360.408942                0.008656           0.478163            2       True        173
78         LightGBM_BAG_L2/T41  -70.928971       2.170843  360.327948                0.007380           0.397169            2       True        175
79         LightGBM_BAG_L3/T18  -71.408163       4.083952  662.626612                0.006305           0.329774            3       True        218
80         LightGBM_BAG_L2/T44  -71.753892       2.171257  360.419116                0.007794           0.488338            2       True        178
81         LightGBM_BAG_L2/T18  -71.865931       2.217102  370.344557                0.053638          10.413778            2       True        152
82         LightGBM_BAG_L1/T81  -73.609044       0.015997    0.307613                0.015997           0.307613            1       True         81
83         LightGBM_BAG_L2/T35  -74.451067       2.171357  360.434605                0.007894           0.503826            2       True        169
84         LightGBM_BAG_L1/T62  -75.045088       0.007167    0.230537                0.007167           0.230537            1       True         62
85          LightGBM_BAG_L3/T1  -76.695278       4.132939  672.718109                0.055292          10.421272            3       True        201
86          LightGBM_BAG_L3/T3  -76.801488       4.147330  672.829674                0.069683          10.532836            3       True        203
87         LightGBM_BAG_L3/T20  -76.843757       4.086789  662.622042                0.009142           0.325205            3       True        220
88          LightGBM_BAG_L2/T3  -77.081298       2.232345  370.779327                0.068881          10.848548            2       True        137
89          LightGBM_BAG_L2/T1  -77.148745       2.223167  371.448302                0.059703          11.517523            2       True        135
90         LightGBM_BAG_L2/T20  -77.280486       2.229053  370.134429                0.065590          10.203650            2       True        154
91        LightGBM_BAG_L1/T111  -78.787486       0.007246    0.230157                0.007246           0.230157            1       True        111
92         LightGBM_BAG_L1/T35  -82.713341       0.056009    9.594652                0.056009           9.594652            1       True         35
93         LightGBM_BAG_L1/T44  -83.723886       0.008006    0.233032                0.008006           0.233032            1       True         44
94         LightGBM_BAG_L1/T65  -84.805025       0.038642    0.487516                0.038642           0.487516            1       True         65
95         LightGBM_BAG_L1/T39  -84.981571       0.006960    0.222264                0.006960           0.222264            1       True         39
96         LightGBM_BAG_L3/T19  -85.311924       4.083980  662.631718                0.006332           0.334881            3       True        219
97         LightGBM_BAG_L2/T19  -85.810719       2.226915  371.381443                0.063451          11.450664            2       True        153
98         LightGBM_BAG_L2/T38  -86.272226       2.170388  360.343070                0.006925           0.412291            2       True        172
99         LightGBM_BAG_L3/T29  -86.564560       4.086278  662.656795                0.008631           0.359957            3       True        229
100        LightGBM_BAG_L2/T29  -87.174273       2.170688  360.390271                0.007225           0.459492            2       True        163
101        LightGBM_BAG_L2/T33  -88.771423       2.174581  360.306565                0.011117           0.375786            2       True        167
102        LightGBM_BAG_L2/T43  -88.974427       2.171088  360.281029                0.007625           0.350250            2       True        177
103        LightGBM_BAG_L1/T64  -89.865774       0.006758    0.242683                0.006758           0.242683            1       True         64
104         LightGBM_BAG_L1/T3  -89.980400       0.055822    9.343952                0.055822           9.343952            1       True          3
105        LightGBM_BAG_L1/T41  -92.297995       0.007127    0.216170                0.007127           0.216170            1       True         41
106        LightGBM_BAG_L1/T66  -93.100964       0.013462    0.412814                0.013462           0.412814            1       True         66
107        LightGBM_BAG_L1/T59  -93.420888       0.007372    0.225325                0.007372           0.225325            1       True         59
108         LightGBM_BAG_L1/T1  -95.246392       0.044828   10.002664                0.044828          10.002664            1       True          1
109        LightGBM_BAG_L1/T23  -96.782527       0.050156   10.137167                0.050156          10.137167            1       True         23
110   NeuralNetTorch_BAG_L1/T6  -96.982748       0.034503    1.504171                0.034503           1.504171            1       True        121
111        LightGBM_BAG_L1/T85  -97.438360       0.012075    0.235161                0.012075           0.235161            1       True         85
112  NeuralNetTorch_BAG_L1/T15  -97.615502       0.042824    2.215915                0.042824           2.215915            1       True        130
113        LightGBM_BAG_L1/T18  -98.233263       0.050075    9.483726                0.050075           9.483726            1       True         18
114  NeuralNetTorch_BAG_L2/T13  -98.269755       2.261712  363.456774                0.098249           3.525995            2       True        199
115        LightGBM_BAG_L1/T57  -98.830000       0.006837    0.217807                0.006837           0.217807            1       True         57
116  NeuralNetTorch_BAG_L1/T11  -99.746162       0.041610    1.879334                0.041610           1.879334            1       True        126
117        LightGBM_BAG_L1/T22  -99.915800       0.049458    9.555924                0.049458           9.555924            1       True         22
118       LightGBM_BAG_L1/T104 -100.469381       0.005826    0.210403                0.005826           0.210403            1       True        104
119        LightGBM_BAG_L1/T58 -100.567002       0.006989    0.222640                0.006989           0.222640            1       True         58
120        LightGBM_BAG_L1/T38 -101.213383       0.006991    0.215770                0.006991           0.215770            1       True         38
121        LightGBM_BAG_L1/T20 -101.277845       0.055207   10.566669                0.055207          10.566669            1       True         20
122        LightGBM_BAG_L1/T19 -102.742899       0.057548    9.555108                0.057548           9.555108            1       True         19
123   NeuralNetTorch_BAG_L1/T4 -102.789106       0.030295    1.222421                0.030295           1.222421            1       True        119
124        LightGBM_BAG_L1/T29 -105.025974       0.055869   10.453753                0.055869          10.453753            1       True         29
125   NeuralNetTorch_BAG_L1/T2 -105.421228       0.032702    1.639454                0.032702           1.639454            1       True        117
126  NeuralNetTorch_BAG_L1/T12 -105.791639       0.028919    0.860202                0.028919           0.860202            1       True        127
127  NeuralNetTorch_BAG_L1/T16 -109.050418       0.047965    2.855595                0.047965           2.855595            1       True        131
128        LightGBM_BAG_L1/T33 -111.545426       0.051340    9.889948                0.051340           9.889948            1       True         33
129        LightGBM_BAG_L3/T14 -111.744829       4.084137  662.607153                0.006490           0.310315            3       True        214
130        LightGBM_BAG_L2/T14 -112.637071       2.270706  370.338713                0.107242          10.407934            2       True        148
131        LightGBM_BAG_L1/T87 -113.480674       0.006609    0.222663                0.006609           0.222663            1       True         87
132        LightGBM_BAG_L1/T88 -114.665779       0.006663    0.223322                0.006663           0.223322            1       True         88
133        LightGBM_BAG_L2/T42 -116.268409       2.171167  360.416349                0.007703           0.485570            2       True        176
134        LightGBM_BAG_L1/T92 -116.418050       0.007560    0.240565                0.007560           0.240565            1       True         92
135        LightGBM_BAG_L2/T50 -119.415014       2.170832  360.369703                0.007369           0.438924            2       True        184
136        LightGBM_BAG_L1/T43 -119.999745       0.005975    0.202012                0.005975           0.202012            1       True         43
137   NeuralNetTorch_BAG_L1/T3 -121.376442       0.034544    1.368851                0.034544           1.368851            1       True        118
138        LightGBM_BAG_L2/T40 -122.155652       2.171247  360.404101                0.007783           0.473322            2       True        174
139        LightGBM_BAG_L1/T75 -123.368369       0.007195    0.239283                0.007195           0.239283            1       True         75
140   NeuralNetTorch_BAG_L1/T5 -124.831996       0.035971    2.305893                0.035971           2.305893            1       True        120
141       LightGBM_BAG_L1/T102 -124.923122       0.006633    0.226857                0.006633           0.226857            1       True        102
142        LightGBM_BAG_L1/T63 -125.976026       0.006454    0.233655                0.006454           0.233655            1       True         63
143       LightGBM_BAG_L1/T108 -126.366307       0.006500    0.215496                0.006500           0.215496            1       True        108
144        LightGBM_BAG_L1/T42 -127.379753       0.007006    0.232998                0.007006           0.232998            1       True         42
145   NeuralNetTorch_BAG_L1/T9 -127.559371       0.035177    1.199696                0.035177           1.199696            1       True        124
146  NeuralNetTorch_BAG_L1/T10 -127.820613       0.037171    1.223811                0.037171           1.223811            1       True        125
147        LightGBM_BAG_L1/T50 -128.712555       0.006351    0.220665                0.006351           0.220665            1       True         50
148  NeuralNetTorch_BAG_L1/T17 -129.014777       0.037956    1.146091                0.037956           1.146091            1       True        132
149         LightGBM_BAG_L3/T6 -131.273761       4.153743  677.279923                0.076096          14.983086            3       True        206
150         LightGBM_BAG_L2/T6 -131.373153       2.229738  370.811903                0.066275          10.881124            2       True        140
151        LightGBM_BAG_L1/T14 -132.465502       0.052489    9.055940                0.052489           9.055940            1       True         14
152   NeuralNetTorch_BAG_L1/T8 -132.917200       0.026577    0.992594                0.026577           0.992594            1       True        123
153        LightGBM_BAG_L3/T21 -133.883575       4.084583  662.666624                0.006936           0.369787            3       True        221
154        LightGBM_BAG_L2/T21 -134.467303       2.225352  370.430907                0.061889          10.500129            2       True        155
155         LightGBM_BAG_L1/T6 -134.945865       0.056398    9.385328                0.056398           9.385328            1       True          6
156        LightGBM_BAG_L1/T40 -135.250951       0.006876    0.240702                0.006876           0.240702            1       True         40
157  NeuralNetTorch_BAG_L1/T13 -135.901553       0.056993    3.418016                0.056993           3.418016            1       True        128
158        LightGBM_BAG_L1/T91 -136.476828       0.006179    0.210755                0.006179           0.210755            1       True         91
159        LightGBM_BAG_L3/T26 -136.779011       4.084606  662.686901                0.006958           0.390064            3       True        226
160       LightGBM_BAG_L1/T103 -137.338684       0.006614    0.235855                0.006614           0.235855            1       True        103
161        LightGBM_BAG_L2/T26 -137.417098       2.233890  371.101029                0.070426          11.170250            2       True        160
162       LightGBM_BAG_L1/T110 -138.439960       0.006432    0.202126                0.006432           0.202126            1       True        110
163       LightGBM_BAG_L1/T112 -138.629973       0.007081    0.215525                0.007081           0.215525            1       True        112
164   NeuralNetTorch_BAG_L1/T1 -138.792950       0.031776    0.941310                0.031776           0.941310            1       True        116
165  NeuralNetTorch_BAG_L1/T14 -140.058903       0.025222    0.996935                0.025222           0.996935            1       True        129
166        LightGBM_BAG_L3/T28 -140.837839       4.084552  662.632007                0.006905           0.335170            3       True        228
167        LightGBM_BAG_L3/T11 -141.465351       4.138753  672.961007                0.061106          10.664170            3       True        211
168        LightGBM_BAG_L1/T21 -141.504419       0.054090   10.074440                0.054090          10.074440            1       True         21
169        LightGBM_BAG_L2/T31 -141.547077       2.171096  360.377096                0.007633           0.446317            2       True        165
170        LightGBM_BAG_L2/T11 -141.575098       2.234422  370.771731                0.070959          10.840952            2       True        145
171        LightGBM_BAG_L2/T28 -141.614454       2.228727  370.963150                0.065263          11.032371            2       True        162
172        LightGBM_BAG_L1/T68 -142.626323       0.008566    0.307189                0.008566           0.307189            1       True         68
173  NeuralNetTorch_BAG_L1/T18 -143.883466       0.045452    2.756621                0.045452           2.756621            1       True        133
174        LightGBM_BAG_L1/T31 -146.321359       0.065132   10.546894                0.065132          10.546894            1       True         31
175        LightGBM_BAG_L1/T70 -146.476142       0.007033    0.251943                0.007033           0.251943            1       True         70
176        LightGBM_BAG_L3/T13 -146.545146       4.149428  672.643589                0.071781          10.346751            3       True        213
177        LightGBM_BAG_L2/T13 -146.616564       2.236465  370.694702                0.073001          10.763923            2       True        147
178        LightGBM_BAG_L1/T84 -146.830452       0.006954    0.232655                0.006954           0.232655            1       True         84
179        LightGBM_BAG_L1/T26 -146.874510       0.058922   10.196182                0.058922          10.196182            1       True         26
180   NeuralNetTorch_BAG_L1/T7 -148.956213       0.025508    1.118420                0.025508           1.118420            1       True        122
181        LightGBM_BAG_L1/T11 -149.408781       0.057441    9.997264                0.057441           9.997264            1       True         11
182        LightGBM_BAG_L1/T28 -150.190487       0.053971    9.465527                0.053971           9.465527            1       True         28
183        LightGBM_BAG_L1/T55 -150.651261       0.008810    0.202813                0.008810           0.202813            1       True         55
184        LightGBM_BAG_L1/T99 -151.300289       0.007069    0.224231                0.007069           0.224231            1       True         99
185        LightGBM_BAG_L2/T49 -152.457810       2.171077  360.371604                0.007613           0.440825            2       True        183
186        LightGBM_BAG_L2/T32 -152.571604       2.170635  360.352623                0.007172           0.421844            2       True        166
187        LightGBM_BAG_L1/T37 -152.964503       0.047522    9.946213                0.047522           9.946213            1       True         37
188        LightGBM_BAG_L1/T13 -154.038604       0.058502    9.197129                0.058502           9.197129            1       True         13
189        LightGBM_BAG_L1/T56 -154.045318       0.006758    0.231108                0.006758           0.231108            1       True         56
190        LightGBM_BAG_L2/T48 -154.113926       2.171930  360.288404                0.008466           0.357625            2       True        182
191       LightGBM_BAG_L1/T105 -154.218162       0.008882    0.214792                0.008882           0.214792            1       True        105
192        LightGBM_BAG_L1/T86 -154.402871       0.006605    0.233217                0.006605           0.233217            1       True         86
193        LightGBM_BAG_L2/T45 -154.764457       2.172047  360.342645                0.008584           0.411866            2       True        179
194        LightGBM_BAG_L3/T12 -155.261240       4.141278  672.574313                0.063630          10.277476            3       True        212
195        LightGBM_BAG_L3/T16 -155.273360       4.084779  662.671921                0.007131           0.375083            3       True        216
196        LightGBM_BAG_L2/T12 -155.365703       2.257148  370.259859                0.093684          10.329080            2       True        146
197        LightGBM_BAG_L2/T16 -155.876853       2.237416  371.102615                0.073953          11.171836            2       True        150
198        LightGBM_BAG_L1/T53 -156.615881       0.007701    0.218027                0.007701           0.218027            1       True         53
199         LightGBM_BAG_L3/T9 -156.711372       4.145249  673.390725                0.067601          11.093887            3       True        209
200         LightGBM_BAG_L2/T9 -156.791000       2.227316  370.887554                0.063853          10.956775            2       True        143
201        LightGBM_BAG_L1/T82 -156.942402       0.006402    0.295640                0.006402           0.295640            1       True         82
202        LightGBM_BAG_L1/T32 -157.555922       0.066901    9.766403                0.066901           9.766403            1       True         32
203        LightGBM_BAG_L1/T74 -158.135111       0.007795    0.225794                0.007795           0.225794            1       True         74
204        LightGBM_BAG_L1/T73 -158.378232       0.006136    0.211785                0.006136           0.211785            1       True         73
205        LightGBM_BAG_L1/T49 -159.453823       0.008553    0.222695                0.008553           0.222695            1       True         49
206         LightGBM_BAG_L1/T9 -160.209171       0.053844    9.100260                0.053844           9.100260            1       True          9
207        LightGBM_BAG_L3/T27 -160.277604       4.085126  662.662751                0.007478           0.365914            3       True        227
208        LightGBM_BAG_L3/T15 -160.539259       4.084344  662.624089                0.006697           0.327252            3       True        215
209        LightGBM_BAG_L1/T12 -160.869643       0.063613    9.382915                0.063613           9.382915            1       True         12
210        LightGBM_BAG_L2/T27 -160.914226       2.240627  370.398039                0.077163          10.467260            2       True        161
211        LightGBM_BAG_L1/T16 -161.096862       0.064341   10.561400                0.064341          10.561400            1       True         16
212        LightGBM_BAG_L2/T15 -161.265009       2.221536  370.144864                0.058072          10.214085            2       True        149
213         LightGBM_BAG_L3/T4 -161.370641       4.138125  672.768961                0.060478          10.472124            3       True        204
214         LightGBM_BAG_L2/T4 -161.414420       2.222565  370.875491                0.059102          10.944712            2       True        138
215        LightGBM_BAG_L2/T52 -161.779784       2.171278  360.352561                0.007815           0.421782            2       True        186
216        LightGBM_BAG_L3/T25 -161.975775       4.084645  662.669852                0.006998           0.373014            3       True        225
217        LightGBM_BAG_L3/T17 -162.022256       4.084596  662.695612                0.006949           0.398774            3       True        217
218        LightGBM_BAG_L1/T77 -162.220783       0.006598    0.219930                0.006598           0.219930            1       True         77
219        LightGBM_BAG_L2/T25 -162.622067       2.222416  371.083233                0.058953          11.152454            2       True        159
220        LightGBM_BAG_L2/T17 -162.622413       2.226909  371.667764                0.063446          11.736985            2       True        151
221        LightGBM_BAG_L1/T45 -163.320324       0.006821    0.216607                0.006821           0.216607            1       True         45
222       LightGBM_BAG_L1/T101 -163.630065       0.007937    0.223276                0.007937           0.223276            1       True        101
223        LightGBM_BAG_L1/T48 -163.765306       0.006855    0.215949                0.006855           0.215949            1       True         48
224        LightGBM_BAG_L1/T80 -164.850379       0.017365    0.269749                0.017365           0.269749            1       True         80
225        LightGBM_BAG_L1/T96 -164.886155       0.006708    0.209600                0.006708           0.209600            1       True         96
226        LightGBM_BAG_L1/T27 -165.244724       0.058286   10.115333                0.058286          10.115333            1       True         27
227         LightGBM_BAG_L1/T4 -165.428965       0.054300    9.584543                0.054300           9.584543            1       True          4
228        LightGBM_BAG_L1/T25 -165.738197       0.055545    9.386972                0.055545           9.386972            1       True         25
229        LightGBM_BAG_L1/T79 -165.816201       0.009422    0.230140                0.009422           0.230140            1       True         79
230        LightGBM_BAG_L1/T15 -165.839770       0.059432   10.260672                0.059432          10.260672            1       True         15
231        LightGBM_BAG_L1/T17 -166.258117       0.060778    9.649885                0.060778           9.649885            1       True         17
232        LightGBM_BAG_L1/T95 -166.662518       0.007148    0.225159                0.007148           0.225159            1       True         95
233        LightGBM_BAG_L1/T52 -168.978088       0.007133    0.225075                0.007133           0.225075            1       True         52
234        LightGBM_BAG_L1/T98 -169.867051       0.007392    0.201504                0.007392           0.201504            1       True         98
235        LightGBM_BAG_L1/T97 -171.494836       0.006243    0.206049                0.006243           0.206049            1       True         97
236       LightGBM_BAG_L1/T115 -180.546931       0.006215    0.189158                0.006215           0.189158            1       True        115
Number of models trained: 237
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_TabularNeuralNetTorch'}
Bagging used: True  (with 6 folds)
Multi-layer stack-ensembling used: True  (with 4 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', [])  : 2 | ['season', 'weather']
('float', [])     : 3 | ['temp', 'atemp', 'windspeed']
('int', [])       : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_134143/SummaryOfModels.html
*** End of fit() summary ***
Out[80]:
{'model_types': {'LightGBM_BAG_L1/T1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T3': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T4': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T5': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T6': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T7': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T8': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T9': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T10': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T11': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T12': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T13': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T14': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T15': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T16': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T17': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T18': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T19': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T20': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T21': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T22': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T23': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T24': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T25': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T26': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T27': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T28': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T29': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T30': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T31': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T32': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T33': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T34': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T35': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T36': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T37': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T38': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T39': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T40': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T41': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T42': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T43': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T44': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T45': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T46': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T47': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T48': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T49': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T50': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T51': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T52': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T53': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T54': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T55': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T56': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T57': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T58': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T59': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T60': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T61': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T62': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T63': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T64': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T65': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T66': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T67': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T68': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T69': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T70': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T71': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T72': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T73': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T74': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T75': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T76': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T77': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T78': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T79': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T80': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T81': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T82': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T83': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T84': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T85': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T86': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T87': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T88': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T89': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T90': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T91': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T92': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T93': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T94': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T95': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T96': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T97': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T98': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T99': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T100': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T101': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T102': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T103': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T104': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T105': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T106': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T107': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T108': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T109': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T110': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T111': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T112': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T113': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T114': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L1/T115': 'StackerEnsembleModel_LGB',
  'NeuralNetTorch_BAG_L1/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T7': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T8': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T9': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T10': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T11': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T12': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T13': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T14': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T15': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T16': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T17': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L1/T18': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'WeightedEnsemble_L2': 'WeightedEnsembleModel',
  'LightGBM_BAG_L2/T1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T3': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T4': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T5': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T6': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T7': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T8': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T9': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T10': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T11': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T12': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T13': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T14': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T15': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T16': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T17': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T18': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T19': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T20': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T21': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T22': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T23': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T24': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T25': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T26': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T27': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T28': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T29': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T30': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T31': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T32': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T33': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T34': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T35': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T36': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T37': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T38': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T39': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T40': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T41': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T42': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T43': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T44': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T45': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T46': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T47': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T48': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T49': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T50': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T51': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L2/T52': 'StackerEnsembleModel_LGB',
  'NeuralNetTorch_BAG_L2/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T7': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T8': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T9': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T10': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T11': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T12': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L2/T13': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'WeightedEnsemble_L3': 'WeightedEnsembleModel',
  'LightGBM_BAG_L3/T1': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T2': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T3': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T4': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T5': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T6': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T7': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T8': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T9': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T10': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T11': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T12': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T13': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T14': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T15': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T16': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T17': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T18': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T19': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T20': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T21': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T22': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T23': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T24': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T25': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T26': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T27': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T28': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T29': 'StackerEnsembleModel_LGB',
  'LightGBM_BAG_L3/T30': 'StackerEnsembleModel_LGB',
  'NeuralNetTorch_BAG_L3/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L3/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L3/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L3/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L3/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'NeuralNetTorch_BAG_L3/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
  'WeightedEnsemble_L4': 'WeightedEnsembleModel'},
 'model_performance': {'LightGBM_BAG_L1/T1': -95.24639164559781,
  'LightGBM_BAG_L1/T2': -70.43029457979837,
  'LightGBM_BAG_L1/T3': -89.98039976556805,
  'LightGBM_BAG_L1/T4': -165.4289650418463,
  'LightGBM_BAG_L1/T5': -62.14989960908567,
  'LightGBM_BAG_L1/T6': -134.94586520147033,
  'LightGBM_BAG_L1/T7': -61.43082287932244,
  'LightGBM_BAG_L1/T8': -61.59581820619491,
  'LightGBM_BAG_L1/T9': -160.20917072309442,
  'LightGBM_BAG_L1/T10': -46.9856225471341,
  'LightGBM_BAG_L1/T11': -149.40878138097298,
  'LightGBM_BAG_L1/T12': -160.8696430394228,
  'LightGBM_BAG_L1/T13': -154.0386043723983,
  'LightGBM_BAG_L1/T14': -132.4655023288866,
  'LightGBM_BAG_L1/T15': -165.8397699719795,
  'LightGBM_BAG_L1/T16': -161.09686164048298,
  'LightGBM_BAG_L1/T17': -166.2581169684571,
  'LightGBM_BAG_L1/T18': -98.23326332950049,
  'LightGBM_BAG_L1/T19': -102.74289872874851,
  'LightGBM_BAG_L1/T20': -101.27784472431108,
  'LightGBM_BAG_L1/T21': -141.50441866062513,
  'LightGBM_BAG_L1/T22': -99.91580049713026,
  'LightGBM_BAG_L1/T23': -96.782526933767,
  'LightGBM_BAG_L1/T24': -48.81178173808668,
  'LightGBM_BAG_L1/T25': -165.73819684652204,
  'LightGBM_BAG_L1/T26': -146.87451027496323,
  'LightGBM_BAG_L1/T27': -165.24472378003523,
  'LightGBM_BAG_L1/T28': -150.1904867526079,
  'LightGBM_BAG_L1/T29': -105.02597407360135,
  'LightGBM_BAG_L1/T30': -59.30136946794056,
  'LightGBM_BAG_L1/T31': -146.32135936203926,
  'LightGBM_BAG_L1/T32': -157.55592226067927,
  'LightGBM_BAG_L1/T33': -111.54542573676088,
  'LightGBM_BAG_L1/T34': -67.21083671487322,
  'LightGBM_BAG_L1/T35': -82.71334070935556,
  'LightGBM_BAG_L1/T36': -63.95082015638741,
  'LightGBM_BAG_L1/T37': -152.964503489081,
  'LightGBM_BAG_L1/T38': -101.21338320908495,
  'LightGBM_BAG_L1/T39': -84.98157102825932,
  'LightGBM_BAG_L1/T40': -135.2509514475015,
  'LightGBM_BAG_L1/T41': -92.29799521482289,
  'LightGBM_BAG_L1/T42': -127.37975329766945,
  'LightGBM_BAG_L1/T43': -119.99974521147973,
  'LightGBM_BAG_L1/T44': -83.72388625714288,
  'LightGBM_BAG_L1/T45': -163.32032355618733,
  'LightGBM_BAG_L1/T46': -57.49809414956977,
  'LightGBM_BAG_L1/T47': -48.49451171986661,
  'LightGBM_BAG_L1/T48': -163.76530597989898,
  'LightGBM_BAG_L1/T49': -159.45382307965372,
  'LightGBM_BAG_L1/T50': -128.71255538821234,
  'LightGBM_BAG_L1/T51': -56.615567759363756,
  'LightGBM_BAG_L1/T52': -168.97808762258396,
  'LightGBM_BAG_L1/T53': -156.61588112932367,
  'LightGBM_BAG_L1/T54': -67.21208914900417,
  'LightGBM_BAG_L1/T55': -150.6512614160556,
  'LightGBM_BAG_L1/T56': -154.0453184654329,
  'LightGBM_BAG_L1/T57': -98.82999986244965,
  'LightGBM_BAG_L1/T58': -100.56700229444995,
  'LightGBM_BAG_L1/T59': -93.42088791087528,
  'LightGBM_BAG_L1/T60': -54.93844950877705,
  'LightGBM_BAG_L1/T61': -47.8342598007438,
  'LightGBM_BAG_L1/T62': -75.04508785029742,
  'LightGBM_BAG_L1/T63': -125.97602550162962,
  'LightGBM_BAG_L1/T64': -89.86577359914915,
  'LightGBM_BAG_L1/T65': -84.80502456784234,
  'LightGBM_BAG_L1/T66': -93.10096416601286,
  'LightGBM_BAG_L1/T67': -53.10814225956848,
  'LightGBM_BAG_L1/T68': -142.626322689254,
  'LightGBM_BAG_L1/T69': -57.806843896862304,
  'LightGBM_BAG_L1/T70': -146.47614183655418,
  'LightGBM_BAG_L1/T71': -45.056774342604,
  'LightGBM_BAG_L1/T72': -52.3685265689472,
  'LightGBM_BAG_L1/T73': -158.37823179945408,
  'LightGBM_BAG_L1/T74': -158.13511074616008,
  'LightGBM_BAG_L1/T75': -123.3683692203776,
  'LightGBM_BAG_L1/T76': -53.18571314452455,
  'LightGBM_BAG_L1/T77': -162.22078301969637,
  'LightGBM_BAG_L1/T78': -56.88118234973596,
  'LightGBM_BAG_L1/T79': -165.81620058357328,
  'LightGBM_BAG_L1/T80': -164.85037888119328,
  'LightGBM_BAG_L1/T81': -73.6090444350346,
  'LightGBM_BAG_L1/T82': -156.94240191957397,
  'LightGBM_BAG_L1/T83': -55.595622403587825,
  'LightGBM_BAG_L1/T84': -146.83045166745313,
  'LightGBM_BAG_L1/T85': -97.4383602196798,
  'LightGBM_BAG_L1/T86': -154.40287115460052,
  'LightGBM_BAG_L1/T87': -113.48067418370216,
  'LightGBM_BAG_L1/T88': -114.66577916739924,
  'LightGBM_BAG_L1/T89': -46.26141836245038,
  'LightGBM_BAG_L1/T90': -49.46816230332918,
  'LightGBM_BAG_L1/T91': -136.47682848555982,
  'LightGBM_BAG_L1/T92': -116.41804973423352,
  'LightGBM_BAG_L1/T93': -56.80441986734995,
  'LightGBM_BAG_L1/T94': -48.50377851532774,
  'LightGBM_BAG_L1/T95': -166.66251812150614,
  'LightGBM_BAG_L1/T96': -164.88615515887656,
  'LightGBM_BAG_L1/T97': -171.49483594639938,
  'LightGBM_BAG_L1/T98': -169.86705058656818,
  'LightGBM_BAG_L1/T99': -151.30028881687844,
  'LightGBM_BAG_L1/T100': -50.93841165301794,
  'LightGBM_BAG_L1/T101': -163.63006530141763,
  'LightGBM_BAG_L1/T102': -124.92312212400682,
  'LightGBM_BAG_L1/T103': -137.3386843303289,
  'LightGBM_BAG_L1/T104': -100.46938070079096,
  'LightGBM_BAG_L1/T105': -154.2181617756975,
  'LightGBM_BAG_L1/T106': -53.02850026381649,
  'LightGBM_BAG_L1/T107': -45.9894585580412,
  'LightGBM_BAG_L1/T108': -126.36630744696683,
  'LightGBM_BAG_L1/T109': -46.43775853220347,
  'LightGBM_BAG_L1/T110': -138.43996035484594,
  'LightGBM_BAG_L1/T111': -78.7874863886287,
  'LightGBM_BAG_L1/T112': -138.62997280215376,
  'LightGBM_BAG_L1/T113': -50.37309404546691,
  'LightGBM_BAG_L1/T114': -51.16639286146918,
  'LightGBM_BAG_L1/T115': -180.5469309330303,
  'NeuralNetTorch_BAG_L1/T1': -138.79295027526143,
  'NeuralNetTorch_BAG_L1/T2': -105.42122784621523,
  'NeuralNetTorch_BAG_L1/T3': -121.37644216000962,
  'NeuralNetTorch_BAG_L1/T4': -102.78910610336368,
  'NeuralNetTorch_BAG_L1/T5': -124.83199619479261,
  'NeuralNetTorch_BAG_L1/T6': -96.9827482169343,
  'NeuralNetTorch_BAG_L1/T7': -148.9562126758656,
  'NeuralNetTorch_BAG_L1/T8': -132.9172001969884,
  'NeuralNetTorch_BAG_L1/T9': -127.55937130584459,
  'NeuralNetTorch_BAG_L1/T10': -127.82061346927256,
  'NeuralNetTorch_BAG_L1/T11': -99.74616246637306,
  'NeuralNetTorch_BAG_L1/T12': -105.79163892386858,
  'NeuralNetTorch_BAG_L1/T13': -135.9015526539169,
  'NeuralNetTorch_BAG_L1/T14': -140.0589031949081,
  'NeuralNetTorch_BAG_L1/T15': -97.6155020257566,
  'NeuralNetTorch_BAG_L1/T16': -109.05041776392018,
  'NeuralNetTorch_BAG_L1/T17': -129.0147771763527,
  'NeuralNetTorch_BAG_L1/T18': -143.88346597908685,
  'WeightedEnsemble_L2': -46.779286301222996,
  'LightGBM_BAG_L2/T1': -77.14874466928583,
  'LightGBM_BAG_L2/T2': -60.11877898625628,
  'LightGBM_BAG_L2/T3': -77.08129804733285,
  'LightGBM_BAG_L2/T4': -161.41441990765844,
  'LightGBM_BAG_L2/T5': -48.52969904011934,
  'LightGBM_BAG_L2/T6': -131.37315263182788,
  'LightGBM_BAG_L2/T7': -51.709883217749145,
  'LightGBM_BAG_L2/T8': -49.79483815388218,
  'LightGBM_BAG_L2/T9': -156.79099995282576,
  'LightGBM_BAG_L2/T10': -43.93703886087322,
  'LightGBM_BAG_L2/T11': -141.57509800949657,
  'LightGBM_BAG_L2/T12': -155.36570273557263,
  'LightGBM_BAG_L2/T13': -146.6165637385049,
  'LightGBM_BAG_L2/T14': -112.63707051801374,
  'LightGBM_BAG_L2/T15': -161.2650085125938,
  'LightGBM_BAG_L2/T16': -155.87685254432824,
  'LightGBM_BAG_L2/T17': -162.62241250526787,
  'LightGBM_BAG_L2/T18': -71.86593112531361,
  'LightGBM_BAG_L2/T19': -85.81071939841397,
  'LightGBM_BAG_L2/T20': -77.28048599041333,
  'LightGBM_BAG_L2/T21': -134.4673028564128,
  'LightGBM_BAG_L2/T22': -70.15881828725007,
  'LightGBM_BAG_L2/T23': -67.92721831361952,
  'LightGBM_BAG_L2/T24': -43.57096096168988,
  'LightGBM_BAG_L2/T25': -162.62206684887906,
  'LightGBM_BAG_L2/T26': -137.4170980060873,
  'LightGBM_BAG_L2/T27': -160.91422570686024,
  'LightGBM_BAG_L2/T28': -141.61445373488476,
  'LightGBM_BAG_L2/T29': -87.17427349216257,
  'LightGBM_BAG_L2/T30': -48.81104568687166,
  'LightGBM_BAG_L2/T31': -141.54707736112474,
  'LightGBM_BAG_L2/T32': -152.5716044129156,
  'LightGBM_BAG_L2/T33': -88.77142277326983,
  'LightGBM_BAG_L2/T34': -55.749415727507035,
  'LightGBM_BAG_L2/T35': -74.45106679994736,
  'LightGBM_BAG_L2/T36': -49.047245533855175,
  'LightGBM_BAG_L2/T37': -50.42936044068636,
  'LightGBM_BAG_L2/T38': -86.2722259935454,
  'LightGBM_BAG_L2/T39': -70.79153046093575,
  'LightGBM_BAG_L2/T40': -122.15565194437455,
  'LightGBM_BAG_L2/T41': -70.92897124704922,
  'LightGBM_BAG_L2/T42': -116.26840944458378,
  'LightGBM_BAG_L2/T43': -88.97442687485974,
  'LightGBM_BAG_L2/T44': -71.75389161928561,
  'LightGBM_BAG_L2/T45': -154.7644565990468,
  'LightGBM_BAG_L2/T46': -50.78878951892069,
  'LightGBM_BAG_L2/T47': -46.89894348888589,
  'LightGBM_BAG_L2/T48': -154.1139259581854,
  'LightGBM_BAG_L2/T49': -152.4578100263843,
  'LightGBM_BAG_L2/T50': -119.41501439177497,
  'LightGBM_BAG_L2/T51': -46.68115243160784,
  'LightGBM_BAG_L2/T52': -161.77978386743854,
  'NeuralNetTorch_BAG_L2/T1': -52.6742758870963,
  'NeuralNetTorch_BAG_L2/T2': -50.92130314076267,
  'NeuralNetTorch_BAG_L2/T3': -67.27410483933936,
  'NeuralNetTorch_BAG_L2/T4': -53.25029244266176,
  'NeuralNetTorch_BAG_L2/T5': -62.08109018839035,
  'NeuralNetTorch_BAG_L2/T6': -52.60961619644969,
  'NeuralNetTorch_BAG_L2/T7': -52.9500383715647,
  'NeuralNetTorch_BAG_L2/T8': -51.47239244542595,
  'NeuralNetTorch_BAG_L2/T9': -51.12143278692236,
  'NeuralNetTorch_BAG_L2/T10': -60.884651060021945,
  'NeuralNetTorch_BAG_L2/T11': -47.65935488999549,
  'NeuralNetTorch_BAG_L2/T12': -51.94952302764032,
  'NeuralNetTorch_BAG_L2/T13': -98.26975527171821,
  'WeightedEnsemble_L3': -43.46031947621305,
  'LightGBM_BAG_L3/T1': -76.69527796121601,
  'LightGBM_BAG_L3/T2': -60.36364733782896,
  'LightGBM_BAG_L3/T3': -76.80148772241041,
  'LightGBM_BAG_L3/T4': -161.37064127439612,
  'LightGBM_BAG_L3/T5': -48.65244531024017,
  'LightGBM_BAG_L3/T6': -131.2737606093407,
  'LightGBM_BAG_L3/T7': -51.765127695720444,
  'LightGBM_BAG_L3/T8': -49.65663526838185,
  'LightGBM_BAG_L3/T9': -156.7113720097058,
  'LightGBM_BAG_L3/T10': -44.345179837557225,
  'LightGBM_BAG_L3/T11': -141.46535113246236,
  'LightGBM_BAG_L3/T12': -155.26123995998728,
  'LightGBM_BAG_L3/T13': -146.54514588050827,
  'LightGBM_BAG_L3/T14': -111.74482884765649,
  'LightGBM_BAG_L3/T15': -160.53925896752193,
  'LightGBM_BAG_L3/T16': -155.27335951898624,
  'LightGBM_BAG_L3/T17': -162.02225552024152,
  'LightGBM_BAG_L3/T18': -71.40816313640647,
  'LightGBM_BAG_L3/T19': -85.31192368561926,
  'LightGBM_BAG_L3/T20': -76.84375696466381,
  'LightGBM_BAG_L3/T21': -133.88357530976188,
  'LightGBM_BAG_L3/T22': -69.64032723601979,
  'LightGBM_BAG_L3/T23': -67.5269483516493,
  'LightGBM_BAG_L3/T24': -45.75767965426102,
  'LightGBM_BAG_L3/T25': -161.97577535636094,
  'LightGBM_BAG_L3/T26': -136.77901114344402,
  'LightGBM_BAG_L3/T27': -160.27760438219826,
  'LightGBM_BAG_L3/T28': -140.8378390205732,
  'LightGBM_BAG_L3/T29': -86.56456014475087,
  'LightGBM_BAG_L3/T30': -48.120607886366784,
  'NeuralNetTorch_BAG_L3/T1': -48.586389034691415,
  'NeuralNetTorch_BAG_L3/T2': -50.26716955215357,
  'NeuralNetTorch_BAG_L3/T3': -59.923183703565556,
  'NeuralNetTorch_BAG_L3/T4': -51.11895920431928,
  'NeuralNetTorch_BAG_L3/T5': -52.629777343342866,
  'NeuralNetTorch_BAG_L3/T6': -54.55176483698387,
  'WeightedEnsemble_L4': -44.345179837557225},
 'model_best': 'WeightedEnsemble_L3',
 'model_paths': {'LightGBM_BAG_L1/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T1/',
  'LightGBM_BAG_L1/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T2/',
  'LightGBM_BAG_L1/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T3/',
  'LightGBM_BAG_L1/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T4/',
  'LightGBM_BAG_L1/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T5/',
  'LightGBM_BAG_L1/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T6/',
  'LightGBM_BAG_L1/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T7/',
  'LightGBM_BAG_L1/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T8/',
  'LightGBM_BAG_L1/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T9/',
  'LightGBM_BAG_L1/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T10/',
  'LightGBM_BAG_L1/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T11/',
  'LightGBM_BAG_L1/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T12/',
  'LightGBM_BAG_L1/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T13/',
  'LightGBM_BAG_L1/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T14/',
  'LightGBM_BAG_L1/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T15/',
  'LightGBM_BAG_L1/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T16/',
  'LightGBM_BAG_L1/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T17/',
  'LightGBM_BAG_L1/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T18/',
  'LightGBM_BAG_L1/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T19/',
  'LightGBM_BAG_L1/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T20/',
  'LightGBM_BAG_L1/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T21/',
  'LightGBM_BAG_L1/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T22/',
  'LightGBM_BAG_L1/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T23/',
  'LightGBM_BAG_L1/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T24/',
  'LightGBM_BAG_L1/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T25/',
  'LightGBM_BAG_L1/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T26/',
  'LightGBM_BAG_L1/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T27/',
  'LightGBM_BAG_L1/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T28/',
  'LightGBM_BAG_L1/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T29/',
  'LightGBM_BAG_L1/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T30/',
  'LightGBM_BAG_L1/T31': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T31/',
  'LightGBM_BAG_L1/T32': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T32/',
  'LightGBM_BAG_L1/T33': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T33/',
  'LightGBM_BAG_L1/T34': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T34/',
  'LightGBM_BAG_L1/T35': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T35/',
  'LightGBM_BAG_L1/T36': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T36/',
  'LightGBM_BAG_L1/T37': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T37/',
  'LightGBM_BAG_L1/T38': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T38/',
  'LightGBM_BAG_L1/T39': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T39/',
  'LightGBM_BAG_L1/T40': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T40/',
  'LightGBM_BAG_L1/T41': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T41/',
  'LightGBM_BAG_L1/T42': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T42/',
  'LightGBM_BAG_L1/T43': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T43/',
  'LightGBM_BAG_L1/T44': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T44/',
  'LightGBM_BAG_L1/T45': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T45/',
  'LightGBM_BAG_L1/T46': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T46/',
  'LightGBM_BAG_L1/T47': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T47/',
  'LightGBM_BAG_L1/T48': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T48/',
  'LightGBM_BAG_L1/T49': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T49/',
  'LightGBM_BAG_L1/T50': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T50/',
  'LightGBM_BAG_L1/T51': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T51/',
  'LightGBM_BAG_L1/T52': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T52/',
  'LightGBM_BAG_L1/T53': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T53/',
  'LightGBM_BAG_L1/T54': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T54/',
  'LightGBM_BAG_L1/T55': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T55/',
  'LightGBM_BAG_L1/T56': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T56/',
  'LightGBM_BAG_L1/T57': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T57/',
  'LightGBM_BAG_L1/T58': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T58/',
  'LightGBM_BAG_L1/T59': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T59/',
  'LightGBM_BAG_L1/T60': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T60/',
  'LightGBM_BAG_L1/T61': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T61/',
  'LightGBM_BAG_L1/T62': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T62/',
  'LightGBM_BAG_L1/T63': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T63/',
  'LightGBM_BAG_L1/T64': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T64/',
  'LightGBM_BAG_L1/T65': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T65/',
  'LightGBM_BAG_L1/T66': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T66/',
  'LightGBM_BAG_L1/T67': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T67/',
  'LightGBM_BAG_L1/T68': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T68/',
  'LightGBM_BAG_L1/T69': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T69/',
  'LightGBM_BAG_L1/T70': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T70/',
  'LightGBM_BAG_L1/T71': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T71/',
  'LightGBM_BAG_L1/T72': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T72/',
  'LightGBM_BAG_L1/T73': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T73/',
  'LightGBM_BAG_L1/T74': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T74/',
  'LightGBM_BAG_L1/T75': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T75/',
  'LightGBM_BAG_L1/T76': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T76/',
  'LightGBM_BAG_L1/T77': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T77/',
  'LightGBM_BAG_L1/T78': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T78/',
  'LightGBM_BAG_L1/T79': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T79/',
  'LightGBM_BAG_L1/T80': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T80/',
  'LightGBM_BAG_L1/T81': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T81/',
  'LightGBM_BAG_L1/T82': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T82/',
  'LightGBM_BAG_L1/T83': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T83/',
  'LightGBM_BAG_L1/T84': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T84/',
  'LightGBM_BAG_L1/T85': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T85/',
  'LightGBM_BAG_L1/T86': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T86/',
  'LightGBM_BAG_L1/T87': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T87/',
  'LightGBM_BAG_L1/T88': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T88/',
  'LightGBM_BAG_L1/T89': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T89/',
  'LightGBM_BAG_L1/T90': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T90/',
  'LightGBM_BAG_L1/T91': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T91/',
  'LightGBM_BAG_L1/T92': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T92/',
  'LightGBM_BAG_L1/T93': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T93/',
  'LightGBM_BAG_L1/T94': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T94/',
  'LightGBM_BAG_L1/T95': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T95/',
  'LightGBM_BAG_L1/T96': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T96/',
  'LightGBM_BAG_L1/T97': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T97/',
  'LightGBM_BAG_L1/T98': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T98/',
  'LightGBM_BAG_L1/T99': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T99/',
  'LightGBM_BAG_L1/T100': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T100/',
  'LightGBM_BAG_L1/T101': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T101/',
  'LightGBM_BAG_L1/T102': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T102/',
  'LightGBM_BAG_L1/T103': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T103/',
  'LightGBM_BAG_L1/T104': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T104/',
  'LightGBM_BAG_L1/T105': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T105/',
  'LightGBM_BAG_L1/T106': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T106/',
  'LightGBM_BAG_L1/T107': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T107/',
  'LightGBM_BAG_L1/T108': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T108/',
  'LightGBM_BAG_L1/T109': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T109/',
  'LightGBM_BAG_L1/T110': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T110/',
  'LightGBM_BAG_L1/T111': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T111/',
  'LightGBM_BAG_L1/T112': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T112/',
  'LightGBM_BAG_L1/T113': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T113/',
  'LightGBM_BAG_L1/T114': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T114/',
  'LightGBM_BAG_L1/T115': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T115/',
  'NeuralNetTorch_BAG_L1/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T1/',
  'NeuralNetTorch_BAG_L1/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T2/',
  'NeuralNetTorch_BAG_L1/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T3/',
  'NeuralNetTorch_BAG_L1/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T4/',
  'NeuralNetTorch_BAG_L1/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T5/',
  'NeuralNetTorch_BAG_L1/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T6/',
  'NeuralNetTorch_BAG_L1/T7': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T7/',
  'NeuralNetTorch_BAG_L1/T8': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T8/',
  'NeuralNetTorch_BAG_L1/T9': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T9/',
  'NeuralNetTorch_BAG_L1/T10': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T10/',
  'NeuralNetTorch_BAG_L1/T11': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T11/',
  'NeuralNetTorch_BAG_L1/T12': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T12/',
  'NeuralNetTorch_BAG_L1/T13': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T13/',
  'NeuralNetTorch_BAG_L1/T14': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T14/',
  'NeuralNetTorch_BAG_L1/T15': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T15/',
  'NeuralNetTorch_BAG_L1/T16': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T16/',
  'NeuralNetTorch_BAG_L1/T17': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T17/',
  'NeuralNetTorch_BAG_L1/T18': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T18/',
  'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L2/',
  'LightGBM_BAG_L2/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T1/',
  'LightGBM_BAG_L2/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T2/',
  'LightGBM_BAG_L2/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T3/',
  'LightGBM_BAG_L2/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T4/',
  'LightGBM_BAG_L2/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T5/',
  'LightGBM_BAG_L2/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T6/',
  'LightGBM_BAG_L2/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T7/',
  'LightGBM_BAG_L2/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T8/',
  'LightGBM_BAG_L2/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T9/',
  'LightGBM_BAG_L2/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T10/',
  'LightGBM_BAG_L2/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T11/',
  'LightGBM_BAG_L2/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T12/',
  'LightGBM_BAG_L2/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T13/',
  'LightGBM_BAG_L2/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T14/',
  'LightGBM_BAG_L2/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T15/',
  'LightGBM_BAG_L2/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T16/',
  'LightGBM_BAG_L2/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T17/',
  'LightGBM_BAG_L2/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T18/',
  'LightGBM_BAG_L2/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T19/',
  'LightGBM_BAG_L2/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T20/',
  'LightGBM_BAG_L2/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T21/',
  'LightGBM_BAG_L2/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T22/',
  'LightGBM_BAG_L2/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T23/',
  'LightGBM_BAG_L2/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T24/',
  'LightGBM_BAG_L2/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T25/',
  'LightGBM_BAG_L2/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T26/',
  'LightGBM_BAG_L2/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T27/',
  'LightGBM_BAG_L2/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T28/',
  'LightGBM_BAG_L2/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T29/',
  'LightGBM_BAG_L2/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T30/',
  'LightGBM_BAG_L2/T31': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T31/',
  'LightGBM_BAG_L2/T32': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T32/',
  'LightGBM_BAG_L2/T33': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T33/',
  'LightGBM_BAG_L2/T34': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T34/',
  'LightGBM_BAG_L2/T35': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T35/',
  'LightGBM_BAG_L2/T36': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T36/',
  'LightGBM_BAG_L2/T37': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T37/',
  'LightGBM_BAG_L2/T38': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T38/',
  'LightGBM_BAG_L2/T39': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T39/',
  'LightGBM_BAG_L2/T40': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T40/',
  'LightGBM_BAG_L2/T41': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T41/',
  'LightGBM_BAG_L2/T42': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T42/',
  'LightGBM_BAG_L2/T43': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T43/',
  'LightGBM_BAG_L2/T44': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T44/',
  'LightGBM_BAG_L2/T45': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T45/',
  'LightGBM_BAG_L2/T46': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T46/',
  'LightGBM_BAG_L2/T47': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T47/',
  'LightGBM_BAG_L2/T48': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T48/',
  'LightGBM_BAG_L2/T49': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T49/',
  'LightGBM_BAG_L2/T50': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T50/',
  'LightGBM_BAG_L2/T51': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T51/',
  'LightGBM_BAG_L2/T52': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T52/',
  'NeuralNetTorch_BAG_L2/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T1/',
  'NeuralNetTorch_BAG_L2/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T2/',
  'NeuralNetTorch_BAG_L2/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T3/',
  'NeuralNetTorch_BAG_L2/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T4/',
  'NeuralNetTorch_BAG_L2/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T5/',
  'NeuralNetTorch_BAG_L2/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T6/',
  'NeuralNetTorch_BAG_L2/T7': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T7/',
  'NeuralNetTorch_BAG_L2/T8': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T8/',
  'NeuralNetTorch_BAG_L2/T9': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T9/',
  'NeuralNetTorch_BAG_L2/T10': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T10/',
  'NeuralNetTorch_BAG_L2/T11': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T11/',
  'NeuralNetTorch_BAG_L2/T12': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T12/',
  'NeuralNetTorch_BAG_L2/T13': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T13/',
  'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L3/',
  'LightGBM_BAG_L3/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T1/',
  'LightGBM_BAG_L3/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T2/',
  'LightGBM_BAG_L3/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T3/',
  'LightGBM_BAG_L3/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T4/',
  'LightGBM_BAG_L3/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T5/',
  'LightGBM_BAG_L3/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T6/',
  'LightGBM_BAG_L3/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T7/',
  'LightGBM_BAG_L3/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T8/',
  'LightGBM_BAG_L3/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T9/',
  'LightGBM_BAG_L3/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T10/',
  'LightGBM_BAG_L3/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T11/',
  'LightGBM_BAG_L3/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T12/',
  'LightGBM_BAG_L3/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T13/',
  'LightGBM_BAG_L3/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T14/',
  'LightGBM_BAG_L3/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T15/',
  'LightGBM_BAG_L3/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T16/',
  'LightGBM_BAG_L3/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T17/',
  'LightGBM_BAG_L3/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T18/',
  'LightGBM_BAG_L3/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T19/',
  'LightGBM_BAG_L3/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T20/',
  'LightGBM_BAG_L3/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T21/',
  'LightGBM_BAG_L3/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T22/',
  'LightGBM_BAG_L3/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T23/',
  'LightGBM_BAG_L3/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T24/',
  'LightGBM_BAG_L3/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T25/',
  'LightGBM_BAG_L3/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T26/',
  'LightGBM_BAG_L3/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T27/',
  'LightGBM_BAG_L3/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T28/',
  'LightGBM_BAG_L3/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T29/',
  'LightGBM_BAG_L3/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T30/',
  'NeuralNetTorch_BAG_L3/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T1/',
  'NeuralNetTorch_BAG_L3/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T2/',
  'NeuralNetTorch_BAG_L3/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T3/',
  'NeuralNetTorch_BAG_L3/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T4/',
  'NeuralNetTorch_BAG_L3/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T5/',
  'NeuralNetTorch_BAG_L3/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T6/',
  'WeightedEnsemble_L4': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L4/'},
 'model_fit_times': {'LightGBM_BAG_L1/T1': 10.002663850784302,
  'LightGBM_BAG_L1/T2': 9.124408483505249,
  'LightGBM_BAG_L1/T3': 9.34395170211792,
  'LightGBM_BAG_L1/T4': 9.584542512893677,
  'LightGBM_BAG_L1/T5': 8.969119787216187,
  'LightGBM_BAG_L1/T6': 9.385327816009521,
  'LightGBM_BAG_L1/T7': 9.292935132980347,
  'LightGBM_BAG_L1/T8': 9.493111610412598,
  'LightGBM_BAG_L1/T9': 9.100260019302368,
  'LightGBM_BAG_L1/T10': 9.394628763198853,
  'LightGBM_BAG_L1/T11': 9.997263669967651,
  'LightGBM_BAG_L1/T12': 9.382915258407593,
  'LightGBM_BAG_L1/T13': 9.197129487991333,
  'LightGBM_BAG_L1/T14': 9.055939674377441,
  'LightGBM_BAG_L1/T15': 10.260672330856323,
  'LightGBM_BAG_L1/T16': 10.561400413513184,
  'LightGBM_BAG_L1/T17': 9.649884939193726,
  'LightGBM_BAG_L1/T18': 9.483726024627686,
  'LightGBM_BAG_L1/T19': 9.555108308792114,
  'LightGBM_BAG_L1/T20': 10.566669464111328,
  'LightGBM_BAG_L1/T21': 10.074440479278564,
  'LightGBM_BAG_L1/T22': 9.555924415588379,
  'LightGBM_BAG_L1/T23': 10.137166500091553,
  'LightGBM_BAG_L1/T24': 9.292906522750854,
  'LightGBM_BAG_L1/T25': 9.386972427368164,
  'LightGBM_BAG_L1/T26': 10.196181774139404,
  'LightGBM_BAG_L1/T27': 10.115333080291748,
  'LightGBM_BAG_L1/T28': 9.465526819229126,
  'LightGBM_BAG_L1/T29': 10.453752517700195,
  'LightGBM_BAG_L1/T30': 9.833702564239502,
  'LightGBM_BAG_L1/T31': 10.546894073486328,
  'LightGBM_BAG_L1/T32': 9.766402959823608,
  'LightGBM_BAG_L1/T33': 9.88994836807251,
  'LightGBM_BAG_L1/T34': 10.786389589309692,
  'LightGBM_BAG_L1/T35': 9.5946524143219,
  'LightGBM_BAG_L1/T36': 9.486711978912354,
  'LightGBM_BAG_L1/T37': 9.946213245391846,
  'LightGBM_BAG_L1/T38': 0.21577024459838867,
  'LightGBM_BAG_L1/T39': 0.22226357460021973,
  'LightGBM_BAG_L1/T40': 0.24070167541503906,
  'LightGBM_BAG_L1/T41': 0.2161695957183838,
  'LightGBM_BAG_L1/T42': 0.23299813270568848,
  'LightGBM_BAG_L1/T43': 0.2020115852355957,
  'LightGBM_BAG_L1/T44': 0.2330324649810791,
  'LightGBM_BAG_L1/T45': 0.21660733222961426,
  'LightGBM_BAG_L1/T46': 0.23194169998168945,
  'LightGBM_BAG_L1/T47': 0.22517013549804688,
  'LightGBM_BAG_L1/T48': 0.21594905853271484,
  'LightGBM_BAG_L1/T49': 0.22269487380981445,
  'LightGBM_BAG_L1/T50': 0.22066521644592285,
  'LightGBM_BAG_L1/T51': 0.20440292358398438,
  'LightGBM_BAG_L1/T52': 0.22507524490356445,
  'LightGBM_BAG_L1/T53': 0.21802711486816406,
  'LightGBM_BAG_L1/T54': 0.2181713581085205,
  'LightGBM_BAG_L1/T55': 0.20281314849853516,
  'LightGBM_BAG_L1/T56': 0.2311084270477295,
  'LightGBM_BAG_L1/T57': 0.21780705451965332,
  'LightGBM_BAG_L1/T58': 0.222639799118042,
  'LightGBM_BAG_L1/T59': 0.2253251075744629,
  'LightGBM_BAG_L1/T60': 0.2102985382080078,
  'LightGBM_BAG_L1/T61': 0.23256182670593262,
  'LightGBM_BAG_L1/T62': 0.23053717613220215,
  'LightGBM_BAG_L1/T63': 0.23365521430969238,
  'LightGBM_BAG_L1/T64': 0.24268293380737305,
  'LightGBM_BAG_L1/T65': 0.487515926361084,
  'LightGBM_BAG_L1/T66': 0.4128139019012451,
  'LightGBM_BAG_L1/T67': 0.24496889114379883,
  'LightGBM_BAG_L1/T68': 0.3071892261505127,
  'LightGBM_BAG_L1/T69': 0.26998162269592285,
  'LightGBM_BAG_L1/T70': 0.25194287300109863,
  'LightGBM_BAG_L1/T71': 0.235917329788208,
  'LightGBM_BAG_L1/T72': 0.21227812767028809,
  'LightGBM_BAG_L1/T73': 0.21178460121154785,
  'LightGBM_BAG_L1/T74': 0.22579431533813477,
  'LightGBM_BAG_L1/T75': 0.23928332328796387,
  'LightGBM_BAG_L1/T76': 0.22036433219909668,
  'LightGBM_BAG_L1/T77': 0.21992969512939453,
  'LightGBM_BAG_L1/T78': 0.22661113739013672,
  'LightGBM_BAG_L1/T79': 0.23014044761657715,
  'LightGBM_BAG_L1/T80': 0.2697489261627197,
  'LightGBM_BAG_L1/T81': 0.30761265754699707,
  'LightGBM_BAG_L1/T82': 0.2956404685974121,
  'LightGBM_BAG_L1/T83': 0.20705318450927734,
  'LightGBM_BAG_L1/T84': 0.23265504837036133,
  'LightGBM_BAG_L1/T85': 0.23516058921813965,
  'LightGBM_BAG_L1/T86': 0.23321747779846191,
  'LightGBM_BAG_L1/T87': 0.22266268730163574,
  'LightGBM_BAG_L1/T88': 0.22332215309143066,
  'LightGBM_BAG_L1/T89': 0.22403383255004883,
  'LightGBM_BAG_L1/T90': 0.20606493949890137,
  'LightGBM_BAG_L1/T91': 0.2107548713684082,
  'LightGBM_BAG_L1/T92': 0.24056529998779297,
  'LightGBM_BAG_L1/T93': 0.22280097007751465,
  'LightGBM_BAG_L1/T94': 0.22036337852478027,
  'LightGBM_BAG_L1/T95': 0.2251594066619873,
  'LightGBM_BAG_L1/T96': 0.20960021018981934,
  'LightGBM_BAG_L1/T97': 0.20604920387268066,
  'LightGBM_BAG_L1/T98': 0.20150423049926758,
  'LightGBM_BAG_L1/T99': 0.22423100471496582,
  'LightGBM_BAG_L1/T100': 0.22870683670043945,
  'LightGBM_BAG_L1/T101': 0.22327589988708496,
  'LightGBM_BAG_L1/T102': 0.22685742378234863,
  'LightGBM_BAG_L1/T103': 0.2358546257019043,
  'LightGBM_BAG_L1/T104': 0.2104032039642334,
  'LightGBM_BAG_L1/T105': 0.21479249000549316,
  'LightGBM_BAG_L1/T106': 0.2222898006439209,
  'LightGBM_BAG_L1/T107': 0.2290949821472168,
  'LightGBM_BAG_L1/T108': 0.21549582481384277,
  'LightGBM_BAG_L1/T109': 0.22116446495056152,
  'LightGBM_BAG_L1/T110': 0.20212554931640625,
  'LightGBM_BAG_L1/T111': 0.23015689849853516,
  'LightGBM_BAG_L1/T112': 0.21552538871765137,
  'LightGBM_BAG_L1/T113': 0.23051166534423828,
  'LightGBM_BAG_L1/T114': 0.22807598114013672,
  'LightGBM_BAG_L1/T115': 0.18915772438049316,
  'NeuralNetTorch_BAG_L1/T1': 0.941309928894043,
  'NeuralNetTorch_BAG_L1/T2': 1.6394541263580322,
  'NeuralNetTorch_BAG_L1/T3': 1.3688507080078125,
  'NeuralNetTorch_BAG_L1/T4': 1.2224206924438477,
  'NeuralNetTorch_BAG_L1/T5': 2.3058929443359375,
  'NeuralNetTorch_BAG_L1/T6': 1.5041708946228027,
  'NeuralNetTorch_BAG_L1/T7': 1.118419885635376,
  'NeuralNetTorch_BAG_L1/T8': 0.9925944805145264,
  'NeuralNetTorch_BAG_L1/T9': 1.1996958255767822,
  'NeuralNetTorch_BAG_L1/T10': 1.223811149597168,
  'NeuralNetTorch_BAG_L1/T11': 1.879333734512329,
  'NeuralNetTorch_BAG_L1/T12': 0.8602015972137451,
  'NeuralNetTorch_BAG_L1/T13': 3.418015718460083,
  'NeuralNetTorch_BAG_L1/T14': 0.9969353675842285,
  'NeuralNetTorch_BAG_L1/T15': 2.2159149646759033,
  'NeuralNetTorch_BAG_L1/T16': 2.8555946350097656,
  'NeuralNetTorch_BAG_L1/T17': 1.1460907459259033,
  'NeuralNetTorch_BAG_L1/T18': 2.7566206455230713,
  'WeightedEnsemble_L2': 0.4757227897644043,
  'LightGBM_BAG_L2/T1': 11.517523050308228,
  'LightGBM_BAG_L2/T2': 10.674018144607544,
  'LightGBM_BAG_L2/T3': 10.848548412322998,
  'LightGBM_BAG_L2/T4': 10.944711923599243,
  'LightGBM_BAG_L2/T5': 10.919796705245972,
  'LightGBM_BAG_L2/T6': 10.881124496459961,
  'LightGBM_BAG_L2/T7': 10.528796434402466,
  'LightGBM_BAG_L2/T8': 10.597357511520386,
  'LightGBM_BAG_L2/T9': 10.956775426864624,
  'LightGBM_BAG_L2/T10': 11.38678765296936,
  'LightGBM_BAG_L2/T11': 10.840951681137085,
  'LightGBM_BAG_L2/T12': 10.329079627990723,
  'LightGBM_BAG_L2/T13': 10.763923406600952,
  'LightGBM_BAG_L2/T14': 10.407933950424194,
  'LightGBM_BAG_L2/T15': 10.214085102081299,
  'LightGBM_BAG_L2/T16': 11.171836376190186,
  'LightGBM_BAG_L2/T17': 11.736985445022583,
  'LightGBM_BAG_L2/T18': 10.413777828216553,
  'LightGBM_BAG_L2/T19': 11.450664281845093,
  'LightGBM_BAG_L2/T20': 10.20365047454834,
  'LightGBM_BAG_L2/T21': 10.500128507614136,
  'LightGBM_BAG_L2/T22': 10.572536706924438,
  'LightGBM_BAG_L2/T23': 10.374094486236572,
  'LightGBM_BAG_L2/T24': 10.308635711669922,
  'LightGBM_BAG_L2/T25': 11.152453899383545,
  'LightGBM_BAG_L2/T26': 11.170250415802002,
  'LightGBM_BAG_L2/T27': 10.467259645462036,
  'LightGBM_BAG_L2/T28': 11.032371282577515,
  'LightGBM_BAG_L2/T29': 0.4594917297363281,
  'LightGBM_BAG_L2/T30': 0.3959805965423584,
  'LightGBM_BAG_L2/T31': 0.4463169574737549,
  'LightGBM_BAG_L2/T32': 0.4218437671661377,
  'LightGBM_BAG_L2/T33': 0.37578582763671875,
  'LightGBM_BAG_L2/T34': 0.4753706455230713,
  'LightGBM_BAG_L2/T35': 0.503826379776001,
  'LightGBM_BAG_L2/T36': 0.3657855987548828,
  'LightGBM_BAG_L2/T37': 0.5009479522705078,
  'LightGBM_BAG_L2/T38': 0.4122910499572754,
  'LightGBM_BAG_L2/T39': 0.478163480758667,
  'LightGBM_BAG_L2/T40': 0.47332215309143066,
  'LightGBM_BAG_L2/T41': 0.3971693515777588,
  'LightGBM_BAG_L2/T42': 0.48556995391845703,
  'LightGBM_BAG_L2/T43': 0.3502504825592041,
  'LightGBM_BAG_L2/T44': 0.48833751678466797,
  'LightGBM_BAG_L2/T45': 0.4118661880493164,
  'LightGBM_BAG_L2/T46': 0.4566938877105713,
  'LightGBM_BAG_L2/T47': 0.43578362464904785,
  'LightGBM_BAG_L2/T48': 0.35762476921081543,
  'LightGBM_BAG_L2/T49': 0.4408249855041504,
  'LightGBM_BAG_L2/T50': 0.43892383575439453,
  'LightGBM_BAG_L2/T51': 0.3358285427093506,
  'LightGBM_BAG_L2/T52': 0.4217817783355713,
  'NeuralNetTorch_BAG_L2/T1': 1.2834010124206543,
  'NeuralNetTorch_BAG_L2/T2': 1.731952428817749,
  'NeuralNetTorch_BAG_L2/T3': 1.9282355308532715,
  'NeuralNetTorch_BAG_L2/T4': 1.477004051208496,
  'NeuralNetTorch_BAG_L2/T5': 2.4433677196502686,
  'NeuralNetTorch_BAG_L2/T6': 1.7685227394104004,
  'NeuralNetTorch_BAG_L2/T7': 1.2231862545013428,
  'NeuralNetTorch_BAG_L2/T8': 1.293245553970337,
  'NeuralNetTorch_BAG_L2/T9': 1.46917724609375,
  'NeuralNetTorch_BAG_L2/T10': 1.6383426189422607,
  'NeuralNetTorch_BAG_L2/T11': 2.257392644882202,
  'NeuralNetTorch_BAG_L2/T12': 0.9296023845672607,
  'NeuralNetTorch_BAG_L2/T13': 3.5259954929351807,
  'WeightedEnsemble_L3': 0.3479495048522949,
  'LightGBM_BAG_L3/T1': 10.421271562576294,
  'LightGBM_BAG_L3/T2': 10.745707035064697,
  'LightGBM_BAG_L3/T3': 10.532835960388184,
  'LightGBM_BAG_L3/T4': 10.472123861312866,
  'LightGBM_BAG_L3/T5': 10.819388151168823,
  'LightGBM_BAG_L3/T6': 14.983085870742798,
  'LightGBM_BAG_L3/T7': 10.5095853805542,
  'LightGBM_BAG_L3/T8': 10.47484827041626,
  'LightGBM_BAG_L3/T9': 11.093887329101562,
  'LightGBM_BAG_L3/T10': 11.26975154876709,
  'LightGBM_BAG_L3/T11': 10.664169788360596,
  'LightGBM_BAG_L3/T12': 10.277475595474243,
  'LightGBM_BAG_L3/T13': 10.346750974655151,
  'LightGBM_BAG_L3/T14': 0.3103153705596924,
  'LightGBM_BAG_L3/T15': 0.327251672744751,
  'LightGBM_BAG_L3/T16': 0.37508320808410645,
  'LightGBM_BAG_L3/T17': 0.3987743854522705,
  'LightGBM_BAG_L3/T18': 0.3297741413116455,
  'LightGBM_BAG_L3/T19': 0.3348805904388428,
  'LightGBM_BAG_L3/T20': 0.32520484924316406,
  'LightGBM_BAG_L3/T21': 0.36978673934936523,
  'LightGBM_BAG_L3/T22': 0.3102102279663086,
  'LightGBM_BAG_L3/T23': 0.3131117820739746,
  'LightGBM_BAG_L3/T24': 0.32816410064697266,
  'LightGBM_BAG_L3/T25': 0.3730144500732422,
  'LightGBM_BAG_L3/T26': 0.3900635242462158,
  'LightGBM_BAG_L3/T27': 0.36591362953186035,
  'LightGBM_BAG_L3/T28': 0.33516955375671387,
  'LightGBM_BAG_L3/T29': 0.3599574565887451,
  'LightGBM_BAG_L3/T30': 0.3373551368713379,
  'NeuralNetTorch_BAG_L3/T1': 1.2312066555023193,
  'NeuralNetTorch_BAG_L3/T2': 1.7005503177642822,
  'NeuralNetTorch_BAG_L3/T3': 1.581681489944458,
  'NeuralNetTorch_BAG_L3/T4': 1.4623653888702393,
  'NeuralNetTorch_BAG_L3/T5': 2.429792881011963,
  'NeuralNetTorch_BAG_L3/T6': 1.9207446575164795,
  'WeightedEnsemble_L4': 0.3485684394836426},
 'model_pred_times': {'LightGBM_BAG_L1/T1': 0.04482841491699219,
  'LightGBM_BAG_L1/T2': 0.06255245208740234,
  'LightGBM_BAG_L1/T3': 0.05582237243652344,
  'LightGBM_BAG_L1/T4': 0.05430006980895996,
  'LightGBM_BAG_L1/T5': 0.0587308406829834,
  'LightGBM_BAG_L1/T6': 0.05639791488647461,
  'LightGBM_BAG_L1/T7': 0.05878496170043945,
  'LightGBM_BAG_L1/T8': 0.05945444107055664,
  'LightGBM_BAG_L1/T9': 0.053844451904296875,
  'LightGBM_BAG_L1/T10': 0.08932852745056152,
  'LightGBM_BAG_L1/T11': 0.057440996170043945,
  'LightGBM_BAG_L1/T12': 0.0636129379272461,
  'LightGBM_BAG_L1/T13': 0.058502197265625,
  'LightGBM_BAG_L1/T14': 0.05248904228210449,
  'LightGBM_BAG_L1/T15': 0.05943155288696289,
  'LightGBM_BAG_L1/T16': 0.06434082984924316,
  'LightGBM_BAG_L1/T17': 0.06077766418457031,
  'LightGBM_BAG_L1/T18': 0.05007529258728027,
  'LightGBM_BAG_L1/T19': 0.057547569274902344,
  'LightGBM_BAG_L1/T20': 0.0552067756652832,
  'LightGBM_BAG_L1/T21': 0.054090261459350586,
  'LightGBM_BAG_L1/T22': 0.04945778846740723,
  'LightGBM_BAG_L1/T23': 0.050156354904174805,
  'LightGBM_BAG_L1/T24': 0.07278990745544434,
  'LightGBM_BAG_L1/T25': 0.05554533004760742,
  'LightGBM_BAG_L1/T26': 0.05892205238342285,
  'LightGBM_BAG_L1/T27': 0.058286428451538086,
  'LightGBM_BAG_L1/T28': 0.053971052169799805,
  'LightGBM_BAG_L1/T29': 0.055869340896606445,
  'LightGBM_BAG_L1/T30': 0.0651853084564209,
  'LightGBM_BAG_L1/T31': 0.06513166427612305,
  'LightGBM_BAG_L1/T32': 0.06690144538879395,
  'LightGBM_BAG_L1/T33': 0.05134010314941406,
  'LightGBM_BAG_L1/T34': 0.06319308280944824,
  'LightGBM_BAG_L1/T35': 0.05600881576538086,
  'LightGBM_BAG_L1/T36': 0.06562376022338867,
  'LightGBM_BAG_L1/T37': 0.04752159118652344,
  'LightGBM_BAG_L1/T38': 0.006990671157836914,
  'LightGBM_BAG_L1/T39': 0.006960391998291016,
  'LightGBM_BAG_L1/T40': 0.0068759918212890625,
  'LightGBM_BAG_L1/T41': 0.007127046585083008,
  'LightGBM_BAG_L1/T42': 0.0070056915283203125,
  'LightGBM_BAG_L1/T43': 0.005974769592285156,
  'LightGBM_BAG_L1/T44': 0.00800633430480957,
  'LightGBM_BAG_L1/T45': 0.0068209171295166016,
  'LightGBM_BAG_L1/T46': 0.0072460174560546875,
  'LightGBM_BAG_L1/T47': 0.007654666900634766,
  'LightGBM_BAG_L1/T48': 0.006855010986328125,
  'LightGBM_BAG_L1/T49': 0.008553028106689453,
  'LightGBM_BAG_L1/T50': 0.006351470947265625,
  'LightGBM_BAG_L1/T51': 0.006451129913330078,
  'LightGBM_BAG_L1/T52': 0.007132768630981445,
  'LightGBM_BAG_L1/T53': 0.007700920104980469,
  'LightGBM_BAG_L1/T54': 0.006682395935058594,
  'LightGBM_BAG_L1/T55': 0.008810043334960938,
  'LightGBM_BAG_L1/T56': 0.0067577362060546875,
  'LightGBM_BAG_L1/T57': 0.006837129592895508,
  'LightGBM_BAG_L1/T58': 0.0069887638092041016,
  'LightGBM_BAG_L1/T59': 0.0073719024658203125,
  'LightGBM_BAG_L1/T60': 0.0072176456451416016,
  'LightGBM_BAG_L1/T61': 0.007396221160888672,
  'LightGBM_BAG_L1/T62': 0.007167339324951172,
  'LightGBM_BAG_L1/T63': 0.006453752517700195,
  'LightGBM_BAG_L1/T64': 0.0067577362060546875,
  'LightGBM_BAG_L1/T65': 0.03864240646362305,
  'LightGBM_BAG_L1/T66': 0.013461828231811523,
  'LightGBM_BAG_L1/T67': 0.008053302764892578,
  'LightGBM_BAG_L1/T68': 0.008565664291381836,
  'LightGBM_BAG_L1/T69': 0.007619619369506836,
  'LightGBM_BAG_L1/T70': 0.007033109664916992,
  'LightGBM_BAG_L1/T71': 0.008518695831298828,
  'LightGBM_BAG_L1/T72': 0.009573221206665039,
  'LightGBM_BAG_L1/T73': 0.006136178970336914,
  'LightGBM_BAG_L1/T74': 0.007794618606567383,
  'LightGBM_BAG_L1/T75': 0.00719451904296875,
  'LightGBM_BAG_L1/T76': 0.007575035095214844,
  'LightGBM_BAG_L1/T77': 0.006597757339477539,
  'LightGBM_BAG_L1/T78': 0.007218837738037109,
  'LightGBM_BAG_L1/T79': 0.00942230224609375,
  'LightGBM_BAG_L1/T80': 0.017364501953125,
  'LightGBM_BAG_L1/T81': 0.01599740982055664,
  'LightGBM_BAG_L1/T82': 0.006402254104614258,
  'LightGBM_BAG_L1/T83': 0.0060558319091796875,
  'LightGBM_BAG_L1/T84': 0.006954193115234375,
  'LightGBM_BAG_L1/T85': 0.012075185775756836,
  'LightGBM_BAG_L1/T86': 0.006605386734008789,
  'LightGBM_BAG_L1/T87': 0.006609201431274414,
  'LightGBM_BAG_L1/T88': 0.006662845611572266,
  'LightGBM_BAG_L1/T89': 0.007159233093261719,
  'LightGBM_BAG_L1/T90': 0.0067920684814453125,
  'LightGBM_BAG_L1/T91': 0.006179094314575195,
  'LightGBM_BAG_L1/T92': 0.007559776306152344,
  'LightGBM_BAG_L1/T93': 0.006592273712158203,
  'LightGBM_BAG_L1/T94': 0.006869792938232422,
  'LightGBM_BAG_L1/T95': 0.007148027420043945,
  'LightGBM_BAG_L1/T96': 0.006707668304443359,
  'LightGBM_BAG_L1/T97': 0.006242513656616211,
  'LightGBM_BAG_L1/T98': 0.007392168045043945,
  'LightGBM_BAG_L1/T99': 0.00706934928894043,
  'LightGBM_BAG_L1/T100': 0.007288217544555664,
  'LightGBM_BAG_L1/T101': 0.007936954498291016,
  'LightGBM_BAG_L1/T102': 0.006632804870605469,
  'LightGBM_BAG_L1/T103': 0.006613969802856445,
  'LightGBM_BAG_L1/T104': 0.005826234817504883,
  'LightGBM_BAG_L1/T105': 0.00888204574584961,
  'LightGBM_BAG_L1/T106': 0.008012771606445312,
  'LightGBM_BAG_L1/T107': 0.007217884063720703,
  'LightGBM_BAG_L1/T108': 0.006499528884887695,
  'LightGBM_BAG_L1/T109': 0.0070536136627197266,
  'LightGBM_BAG_L1/T110': 0.006432056427001953,
  'LightGBM_BAG_L1/T111': 0.007245540618896484,
  'LightGBM_BAG_L1/T112': 0.007080554962158203,
  'LightGBM_BAG_L1/T113': 0.00712132453918457,
  'LightGBM_BAG_L1/T114': 0.01260232925415039,
  'LightGBM_BAG_L1/T115': 0.006214618682861328,
  'NeuralNetTorch_BAG_L1/T1': 0.031775712966918945,
  'NeuralNetTorch_BAG_L1/T2': 0.03270244598388672,
  'NeuralNetTorch_BAG_L1/T3': 0.03454399108886719,
  'NeuralNetTorch_BAG_L1/T4': 0.030295372009277344,
  'NeuralNetTorch_BAG_L1/T5': 0.03597140312194824,
  'NeuralNetTorch_BAG_L1/T6': 0.03450298309326172,
  'NeuralNetTorch_BAG_L1/T7': 0.025508403778076172,
  'NeuralNetTorch_BAG_L1/T8': 0.026576995849609375,
  'NeuralNetTorch_BAG_L1/T9': 0.035176992416381836,
  'NeuralNetTorch_BAG_L1/T10': 0.037171125411987305,
  'NeuralNetTorch_BAG_L1/T11': 0.041609764099121094,
  'NeuralNetTorch_BAG_L1/T12': 0.02891850471496582,
  'NeuralNetTorch_BAG_L1/T13': 0.05699324607849121,
  'NeuralNetTorch_BAG_L1/T14': 0.025221824645996094,
  'NeuralNetTorch_BAG_L1/T15': 0.04282379150390625,
  'NeuralNetTorch_BAG_L1/T16': 0.04796481132507324,
  'NeuralNetTorch_BAG_L1/T17': 0.03795623779296875,
  'NeuralNetTorch_BAG_L1/T18': 0.045452117919921875,
  'WeightedEnsemble_L2': 0.0011477470397949219,
  'LightGBM_BAG_L2/T1': 0.05970311164855957,
  'LightGBM_BAG_L2/T2': 0.07781696319580078,
  'LightGBM_BAG_L2/T3': 0.06888103485107422,
  'LightGBM_BAG_L2/T4': 0.05910181999206543,
  'LightGBM_BAG_L2/T5': 0.06439924240112305,
  'LightGBM_BAG_L2/T6': 0.06627464294433594,
  'LightGBM_BAG_L2/T7': 0.06974911689758301,
  'LightGBM_BAG_L2/T8': 0.06839776039123535,
  'LightGBM_BAG_L2/T9': 0.06385278701782227,
  'LightGBM_BAG_L2/T10': 0.06974506378173828,
  'LightGBM_BAG_L2/T11': 0.07095861434936523,
  'LightGBM_BAG_L2/T12': 0.09368443489074707,
  'LightGBM_BAG_L2/T13': 0.07300138473510742,
  'LightGBM_BAG_L2/T14': 0.10724234580993652,
  'LightGBM_BAG_L2/T15': 0.05807232856750488,
  'LightGBM_BAG_L2/T16': 0.07395267486572266,
  'LightGBM_BAG_L2/T17': 0.0634455680847168,
  'LightGBM_BAG_L2/T18': 0.053638458251953125,
  'LightGBM_BAG_L2/T19': 0.06345129013061523,
  'LightGBM_BAG_L2/T20': 0.06558966636657715,
  'LightGBM_BAG_L2/T21': 0.061888694763183594,
  'LightGBM_BAG_L2/T22': 0.0582273006439209,
  'LightGBM_BAG_L2/T23': 0.05881214141845703,
  'LightGBM_BAG_L2/T24': 0.07249259948730469,
  'LightGBM_BAG_L2/T25': 0.05895256996154785,
  'LightGBM_BAG_L2/T26': 0.07042622566223145,
  'LightGBM_BAG_L2/T27': 0.0771629810333252,
  'LightGBM_BAG_L2/T28': 0.06526303291320801,
  'LightGBM_BAG_L2/T29': 0.007224559783935547,
  'LightGBM_BAG_L2/T30': 0.007968425750732422,
  'LightGBM_BAG_L2/T31': 0.007632732391357422,
  'LightGBM_BAG_L2/T32': 0.007171630859375,
  'LightGBM_BAG_L2/T33': 0.011116981506347656,
  'LightGBM_BAG_L2/T34': 0.008387565612792969,
  'LightGBM_BAG_L2/T35': 0.007893562316894531,
  'LightGBM_BAG_L2/T36': 0.011198282241821289,
  'LightGBM_BAG_L2/T37': 0.009037017822265625,
  'LightGBM_BAG_L2/T38': 0.006924867630004883,
  'LightGBM_BAG_L2/T39': 0.00865626335144043,
  'LightGBM_BAG_L2/T40': 0.007783412933349609,
  'LightGBM_BAG_L2/T41': 0.007379770278930664,
  'LightGBM_BAG_L2/T42': 0.007703304290771484,
  'LightGBM_BAG_L2/T43': 0.007624626159667969,
  'LightGBM_BAG_L2/T44': 0.007793903350830078,
  'LightGBM_BAG_L2/T45': 0.008583784103393555,
  'LightGBM_BAG_L2/T46': 0.009446382522583008,
  'LightGBM_BAG_L2/T47': 0.00810551643371582,
  'LightGBM_BAG_L2/T48': 0.008466482162475586,
  'LightGBM_BAG_L2/T49': 0.007613420486450195,
  'LightGBM_BAG_L2/T50': 0.007368803024291992,
  'LightGBM_BAG_L2/T51': 0.007039070129394531,
  'LightGBM_BAG_L2/T52': 0.007814884185791016,
  'NeuralNetTorch_BAG_L2/T1': 0.07587409019470215,
  'NeuralNetTorch_BAG_L2/T2': 0.03550553321838379,
  'NeuralNetTorch_BAG_L2/T3': 0.07411432266235352,
  'NeuralNetTorch_BAG_L2/T4': 0.06741023063659668,
  'NeuralNetTorch_BAG_L2/T5': 0.2763195037841797,
  'NeuralNetTorch_BAG_L2/T6': 0.0688178539276123,
  'NeuralNetTorch_BAG_L2/T7': 0.07483291625976562,
  'NeuralNetTorch_BAG_L2/T8': 0.07549834251403809,
  'NeuralNetTorch_BAG_L2/T9': 0.08083772659301758,
  'NeuralNetTorch_BAG_L2/T10': 0.07576513290405273,
  'NeuralNetTorch_BAG_L2/T11': 0.09225821495056152,
  'NeuralNetTorch_BAG_L2/T12': 0.026915311813354492,
  'NeuralNetTorch_BAG_L2/T13': 0.09824872016906738,
  'WeightedEnsemble_L3': 0.0008761882781982422,
  'LightGBM_BAG_L3/T1': 0.05529189109802246,
  'LightGBM_BAG_L3/T2': 0.06210684776306152,
  'LightGBM_BAG_L3/T3': 0.06968259811401367,
  'LightGBM_BAG_L3/T4': 0.06047797203063965,
  'LightGBM_BAG_L3/T5': 0.0686500072479248,
  'LightGBM_BAG_L3/T6': 0.0760955810546875,
  'LightGBM_BAG_L3/T7': 0.07187533378601074,
  'LightGBM_BAG_L3/T8': 0.05547356605529785,
  'LightGBM_BAG_L3/T9': 0.06760144233703613,
  'LightGBM_BAG_L3/T10': 0.06429338455200195,
  'LightGBM_BAG_L3/T11': 0.061105966567993164,
  'LightGBM_BAG_L3/T12': 0.06363034248352051,
  'LightGBM_BAG_L3/T13': 0.07178068161010742,
  'LightGBM_BAG_L3/T14': 0.006489753723144531,
  'LightGBM_BAG_L3/T15': 0.006696939468383789,
  'LightGBM_BAG_L3/T16': 0.007131338119506836,
  'LightGBM_BAG_L3/T17': 0.006948947906494141,
  'LightGBM_BAG_L3/T18': 0.006304502487182617,
  'LightGBM_BAG_L3/T19': 0.0063323974609375,
  'LightGBM_BAG_L3/T20': 0.009141921997070312,
  'LightGBM_BAG_L3/T21': 0.006935834884643555,
  'LightGBM_BAG_L3/T22': 0.006928920745849609,
  'LightGBM_BAG_L3/T23': 0.006405830383300781,
  'LightGBM_BAG_L3/T24': 0.007336854934692383,
  'LightGBM_BAG_L3/T25': 0.006997585296630859,
  'LightGBM_BAG_L3/T26': 0.0069582462310791016,
  'LightGBM_BAG_L3/T27': 0.007478475570678711,
  'LightGBM_BAG_L3/T28': 0.00690460205078125,
  'LightGBM_BAG_L3/T29': 0.008630990982055664,
  'LightGBM_BAG_L3/T30': 0.0074939727783203125,
  'NeuralNetTorch_BAG_L3/T1': 0.06948494911193848,
  'NeuralNetTorch_BAG_L3/T2': 0.0382237434387207,
  'NeuralNetTorch_BAG_L3/T3': 0.07168054580688477,
  'NeuralNetTorch_BAG_L3/T4': 0.06579947471618652,
  'NeuralNetTorch_BAG_L3/T5': 0.07570505142211914,
  'NeuralNetTorch_BAG_L3/T6': 0.0714726448059082,
  'WeightedEnsemble_L4': 0.0007472038269042969},
 'num_bag_folds': 6,
 'max_stack_level': 4,
 'model_hyperparams': {'LightGBM_BAG_L1/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T7': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T8': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T9': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T10': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T11': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T12': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T13': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T14': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T15': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T16': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T17': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T18': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T19': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T20': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T21': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T22': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T23': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T24': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T25': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T26': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T27': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T28': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T29': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T30': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T31': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T32': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T33': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T34': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T35': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T36': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T37': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T38': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T39': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T40': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T41': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T42': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T43': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T44': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T45': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T46': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T47': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T48': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T49': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T50': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T51': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T52': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T53': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T54': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T55': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T56': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T57': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T58': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T59': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T60': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T61': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T62': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T63': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T64': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T65': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T66': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T67': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T68': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T69': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T70': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T71': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T72': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T73': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T74': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T75': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T76': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T77': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T78': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T79': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T80': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T81': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T82': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T83': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T84': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T85': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T86': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T87': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T88': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T89': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T90': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T91': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T92': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T93': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T94': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T95': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T96': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T97': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T98': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T99': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T100': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T101': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T102': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T103': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T104': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T105': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T106': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T107': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T108': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T109': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T110': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T111': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T112': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T113': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T114': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L1/T115': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T7': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T8': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T9': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T10': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T11': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T12': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T13': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T14': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T15': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T16': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T17': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L1/T18': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L2': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T7': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T8': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T9': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T10': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T11': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T12': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T13': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T14': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T15': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T16': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T17': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T18': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T19': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T20': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T21': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T22': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T23': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T24': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T25': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T26': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T27': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T28': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T29': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T30': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T31': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T32': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T33': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T34': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T35': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T36': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T37': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T38': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T39': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T40': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T41': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T42': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T43': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T44': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T45': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T46': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T47': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T48': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T49': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T50': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T51': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L2/T52': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T7': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T8': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T9': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T10': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T11': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T12': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L2/T13': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L3': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T7': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T8': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T9': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T10': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T11': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T12': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T13': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T14': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T15': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T16': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T17': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T18': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T19': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T20': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T21': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T22': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T23': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T24': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T25': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T26': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T27': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T28': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T29': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'LightGBM_BAG_L3/T30': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T1': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T2': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T3': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T4': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T5': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'NeuralNetTorch_BAG_L3/T6': {'use_orig_features': True,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True},
  'WeightedEnsemble_L4': {'use_orig_features': False,
   'max_base_models': 25,
   'max_base_models_per_type': 5,
   'save_bag_folds': True}},
 'leaderboard':                     model   score_val  pred_time_val    fit_time  \
 0     WeightedEnsemble_L3  -43.460319       2.306577  381.974152   
 1     LightGBM_BAG_L2/T24  -43.570961       2.235956  370.239415   
 2     LightGBM_BAG_L2/T10  -43.937039       2.233209  371.317567   
 3     LightGBM_BAG_L3/T10  -44.345180       4.141941  673.566589   
 4     WeightedEnsemble_L4  -44.345180       4.142688  673.915158   
 ..                    ...         ...            ...         ...   
 232   LightGBM_BAG_L1/T95 -166.662518       0.007148    0.225159   
 233   LightGBM_BAG_L1/T52 -168.978088       0.007133    0.225075   
 234   LightGBM_BAG_L1/T98 -169.867051       0.007392    0.201504   
 235   LightGBM_BAG_L1/T97 -171.494836       0.006243    0.206049   
 236  LightGBM_BAG_L1/T115 -180.546931       0.006215    0.189158   
 
      pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  \
 0                  0.000876           0.347950            3       True   
 1                  0.072493          10.308636            2       True   
 2                  0.069745          11.386788            2       True   
 3                  0.064293          11.269752            3       True   
 4                  0.000747           0.348568            4       True   
 ..                      ...                ...          ...        ...   
 232                0.007148           0.225159            1       True   
 233                0.007133           0.225075            1       True   
 234                0.007392           0.201504            1       True   
 235                0.006243           0.206049            1       True   
 236                0.006215           0.189158            1       True   
 
      fit_order  
 0          200  
 1          158  
 2          144  
 3          210  
 4          237  
 ..         ...  
 232         95  
 233         52  
 234         98  
 235         97  
 236        115  
 
 [237 rows x 9 columns]}

Create predictions for train dataset

In [81]:
#predictor_new_hpo_v2
predictor_new_hpov2_train = predictor_new_hpo_v2.predict_proba(train)

rmse_new_hpov2 = mean_squared_error(train['count'], predictor_new_hpov2_train, squared=False)
print('The RMSE for predictor new hpo is: ',rmse_new_hpov2)
The RMSE for predictor new hpo is:  35.513254542697794

Create predictions for test dataset

In [82]:
#predictor_new_hpo_v2
predictions_new_hpo_v2 = predictor_new_hpo_v2.predict_proba(test)
In [83]:
submission_new_hpo_v2 = pd.DataFrame({'datetime': submission['datetime'],
                           'count': predictions_new_hpo_v2})

submission_new_hpo_v2.loc[submission_new_hpo_v2['count'] < 0, 'count'] = 0
In [84]:
# Same submitting predictions
submission_new_hpo_v2['count'] = submission_new_hpo_v2['count'].fillna(0).astype(int)
submission_new_hpo_v2['count'] = submission_new_hpo_v2['count'].astype(np.int64)

submission_new_hpo_v2.to_csv("submission_new_hpo_v2.csv", index=False)
In [85]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_hpo_v2.csv -m "new features with hyperparameters v2"
100%|█████████████████████████████████████████| 150k/150k [00:00<00:00, 364kB/s]
Successfully submitted to Bike Sharing Demand
In [86]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName                     date                 description                           status    publicScore  privateScore  
---------------------------  -------------------  ------------------------------------  --------  -----------  ------------  
submission_new_hpo_v2.csv    2022-05-21 14:01:59  new features with hyperparameters v2  complete  0.54039      0.54039       
submission_new_hpo.csv       2022-05-21 12:42:39  new features with hyperparameters     complete  0.47659      0.47659       
submission_new_features.csv  2022-05-21 12:28:51  new features                          complete  0.46603      0.46603       
submission.csv               2022-05-21 12:08:15  first raw submission                  complete  1.82081      1.82081       

New Score of 0.54039

Step 7: Write a Report

Refer to the markdown file for the full report

Creating plots and table for report

In [89]:
# Taking the top model score from each training run and creating a line plot to show improvement
# You can create these in the notebook and save them to PNG or use some other tool (e.g. google sheets, excel)
fig = pd.DataFrame(
    {
        "model": ["initial", "add_features", "hpo", "hpo_v2"],
        "score": [
            predictor.leaderboard(silent=True)['score_val'][0],
            predictor_new_features.leaderboard(silent=True)['score_val'][0],
            predictor_new_hpo.leaderboard(silent=True)['score_val'][0],
            predictor_new_hpo_v2.leaderboard(silent=True)['score_val'][0]],
    }
).plot(x="model", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_train_score.png')
In [90]:
# Take the 3 kaggle scores and creating a line plot to show improvement
fig = pd.DataFrame(
    {
        "test_eval": ["initial", "add_features", "hpo", "hpo_v2"],
        "score": [1.82081, 0.46603,  0.47659, 0.54039]
    }
).plot(x="test_eval", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_test_score.png')

Hyperparameter table

In [91]:
# The 3 hyperparameters we tuned with the kaggle score as the result
pd.DataFrame({
    "model": ["initial", "add_features", "hpo", "hpo_v2"],
    "time": [600, 900, 1200, 1200],
    "num_bag_folds": [8, 8, 5, 6],
    "num_stack_levels": [3, 3, 2, 2],
    "score": [1.82081, 0.46603, 0.47659, 0.54039]
})
Out[91]:
model time num_bag_folds num_stack_levels score
0 initial 600 8 3 1.82081
1 add_features 900 8 3 0.46603
2 hpo 1200 5 2 0.47659
3 hpo_v2 1200 6 2 0.54039